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State Board of Education) for middle school. 
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engineering practices. Using science notebooks, Unified“ as 
questioning strategies, and other approaches, 
students conduct investigations, construct argu- 
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ulate ideas, and assess their own understanding. 
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year through a Teaching Learning Collaborative. =—w 
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Learn more about this multi-year initiative and access evaluation findings as well as instructional 
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Evaluation of the NGSS 
Early Implementers Initiative 


The S. D. Bechtel, Jr. Foundation has commissioned WestEd’s STEM Evaluation 
Unit to evaluate the NGSS Early Implementers Initiative in the eight participating 
public school districts. This independent evaluation is advised by a technical work- 
ing group that includes representatives of the California Department of Education 
and the State Board of Education. Evaluators investigate three main aspects of the 
Initiative's NGSS implementation: 


> districts’ local implementation, 
> implementation support provided by K-12 Alliance, and 


> the resulting science teaching and leadership growth of teachers and admin- 
istrators, as well as student outcomes. 


In addition to this current Report #8, evaluators previously released: 


> The Needle Is Moving in California K-8 Science: Integration with English 
Language Arts, Integration of the Sciences, and Returning Science as a 
K-8 Core Subject (Evaluation Report #1, October 2016) 


The Synergy of Science and English Language Arts: Means and Mutual 
Benefits of Integration (Evaluation Report #2, October 2017) 


Administrators Matter in NGSS Implementation: How School and District 
Leaders Are Making Science Happen (Evaluation Report #3, November 2017) 


Developing District Plans for NGSS Implementation: Preventing Detours 
and Finding Express Lanes on the Journey to Implement the New Science 
Standards (Evaluation Report #4, February 2018) 


Next Generation Science Standards in Practice: Tools and Processes Used by 
the California NGSS Early Implementers (May 2018) 


Making Middle School Science Whole: Transitioning to an Integrated 
Approach to Science Instruction (Evaluation Report #5, October 2018) 


Engaged and Learning Science: How Students Benefit from Next Generation 
Science Standards Teaching (Evaluation Report #6, November 2018) 


Investing in Science Teacher Leadership: Strategies and Impacts in the NGSS 
Early Implementers Initiative (Evaluation Report #7, February 2019) 


This evaluation report describes a central profes- 
sional learning strategy that the California NGSS 
Early Implementers Initiative used to help teach- 
ers effectively transition to the Next Generation 
Science Standards (NGSS). The Initiative's 
approach to lesson study, called a Teaching 
Learning Collaborative (TLC), brings together 
teams of teachers who collaboratively plan, 
teach, critique, and then re-teach NGSS lessons. 
Trained facilitators ensure that participants feel 
professionally safe and supported to experiment 
with the substantial and sometimes daunting 
instructional shifts required by the NGSS. The 
especially strong emphasis on active collaboration 
is why the Initiative uses the term “TLC” rather 
than the more general, commonly used term 
“lesson studies.” 


Initiative leaders chose to focus heavily on TLCs 
because, while teacher professional learning often 
takes place outside of the classroom, TLCs, like all 
lesson studies, provide true hands-on learning ina 
classroom setting where teachers can grapple with 
authentic instructional issues. 


All NGSS Early Implementer districts followed the 
same TLC model during Years 1 through 4 of the 
Initiative. In Year 5, when grant funding began 

to scale back and professional learning became 
less centralized, districts made a variety of modi- 
fications to TLCs to meet their local needs and 
circumstances. 


This report describes: 


> The original TLC model used Initiative-wide in 
Years 1—4, and its benefits 


> District modifications to TLCs in Year 5 


> Feedback from participants about what was 
gained and lost through those modifications 


> Recommendations for using TLCs as NGSS 
professional learning 


Like the entire evaluation series for the NGSS 
Early Implementers Initiative, this report provides 
useful information to school and district admin- 
istrators, leaders of science professional learning, 
and state policymakers. The report is based on an 
extensive amount of data: evaluators’ observation 
of 27 TLCs; responses from a dozen surveys; and 
over 100 interviews with teachers, administra- 
tors, district Project Directors, and K-12 Alliance 
Regional Directors. 


TLCs in Practice 


The TLCs of Years 1 to 4 


Every fall and spring during the first four years 

of the Initiative, TLC teams of three to four 
grade-level teachers, drawn from different schools 
in their districts, convened for a lesson planning 
day and then, one to three weeks later, a teaching 
day. During the planning day, a facilitator led the 
team through a process of developing a lesson to 
be collaboratively taught by the team during a 
single class period. During the teaching day, the 
team co-taught the lesson to students and immedi- 
ately afterward examined student work to debrief 
the lesson’s effectiveness in advancing student 
understanding. The team then made revisions to 
improve the lesson, taught it to another class, and 
held a final debrief that included sharing what 
they each had learned and would take back to their 
classrooms. The report includes a detailed vignette 
of the full planning and teaching days of a grade 6 
TLC observed by an evaluator. 
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Benefits of the Early Implementer TLCs 


Evaluation data show that the TLC experience 
accelerated teachers’ enactment of the pedagogi- 
cal shifts required by the NGSS. More specifically, 
the TLCs: 


> Powerfully engaged teachers in learning, moti- 
vating them to want to implement instruction 
aligned with NGSS 


> Gave teachers detailed, grounded insights into 
what the NGSS standards are — and what it 
means to implement instruction aligned with 
these standards 


> Enhanced the culture of teaching, learning, and 
collaboration in school sites and districts 


About 80 percent of teachers reported that TLCs 
deepened their understanding of each of the 
following core aspects of NGSS “a lot” or “moder- 
ately": science disciplinary core ideas (DCIs), 
science and engineering practices (SEPs), cross- 
cutting concepts (CCCs), and how to use these 
three dimensions to help students understand a 
phenomenon. Further, TLCs also helped the many 
administrators who took advantage of an open 
invitation to observe. For example, when asked 

in a survey, “During the 2017-18 school year, how 
much of an impact did the following have on your 
understanding of NGSS?", 41 percent of principals 
reported that observing a TLC teaching day had 
"A great deal” of impact, and 32 percent said it had 
“Some” impact. 


Modified TLCs in Year 5 


The largest section of the report presents the 
variety of modifications made by districts to TLCs 
when Early Implementer grant funding decreased 
(per the Initiative budget schedule) in Year 5. In 
most districts, there was a need to reduce release 
time for teachers and reliance on substitutes. 
Further, districts in Year 5 aimed to expand 
professional learning to a much larger number 

of teachers. 


All districts retained certain features of the 
original TLC: a facilitator to shepherd the teams 
through the TLC process; an emphasis on collabo- 
ration and support; and some version of planning, 
teaching, and debriefing. In all districts, reviewing 
student work was central to the debrief discus- 
sion. Some facilitators provided lessons to the TLC 
teams or brought suggestions for phenomena or 
lessons in order to reduce planning time. Most 
teachers taught the lessons on their own with their 
own students, bringing student work to the debrief 
sessions. None of the districts conducted TLCs 
exactly as they had in Years 1 through 4. According 
to feedback, the biggest loss in Year 5 was the 
co-teaching element of the original TLC since it 
was such arich learning opportunity. A vignette in 
the report illustrates one district's use of the TLC 
to support both NGSS implementation and integra- 
tion of science and ELA in Year 5. 


Lessons Learned 


> The steep learning curve to understand and 
teach the NGSS as intended necessitates inten- 
sive professional learning opportunities for 
teachers, such as the TLCs. 


> While TLCs are costly and time-intensive, the 
needs they meet and the benefits they deliver 
are high — that is, they require investment 
that pays dividends. 


> Because the instructional changes called for 
by NGSS can be intimidating (particularly for 
elementary teachers), strong attention must be 
paid to the need for a collaborative and nonjudg- 
mental space for teachers to try new things. 


> The real payoff of bringing teachers together 
for lesson planning will not happen unless 
participants also come together to teach, then 
debrief, the lessons. 


> With new instructional materials for science 
available for adoption, it may be tempting to 
assume that the need for professional learning 
is reduced, but administrators and teachers 
should not expect new curricula to stand in for 
professional learning. 
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This report, the eighth in a series of evaluation 
reports about the NGSS Early Implementer 
Initiative, describes how the Initiative’s deep 

and extensive use of lesson studies has yielded 
widespread, powerful NGSS professional learn- 
ing. The Initiative's lesson studies are called 
Teaching Learning Collaboratives, hereafter 
referred to as TLCs (DiRanna, Topps, Cerwin, & 
Gomez-Zwiep, 2009). TLCs held during the school 
year have been one of the two strategies that 
compose the lion's share of the Initiative's profes- 
sional learning efforts, the other being Summer 
Institutes.* The Initiative chose to focus this 
heavily on TLCs because, while most professional 
learning for teachers takes place outside of the 
classroom, TLCs, like all lesson studies, provide 
true hands-on learning in the classroom setting 
where teachers grapple with instructional issues. 
Indeed, researchers have found that the most 
effective professional development for teachers is 
that which occurs closest to the classroom (Cohen 
& Hill, 1998; Grossman et al, 2009; Putnam & 
Borko, 2000). 


TLCs establish intimate teams of Early 
Implementer teachers who come together to 
plan, teach, critique, and reteach an NGSS lesson. 
The experience accelerates teachers’ enactment 
of the pedagogical shifts required by the NGSS. 
Specifically, evaluators learned from Initiative 
participants as well as their own direct observa- 
tions that the TLCs: 


a 


“Just going to professional development is not 


the same as writing and implementing a lesson 
with a team.” [Middle school principal] 


> Powerfully engaged teachers in learning and 
motivated them to want to implement NGSS 


> Gave teachers detailed, grounded insights into 
what the NGSS standards are and what imple- 
menting them means 


> Enhanced the culture of teaching, learning, 
and collaboration in school sites and districts 


During Years 1 to 4, all of the Initiative's Teacher 
Leaders? had two rounds of TLCs per year. The 
TLCs in Years 1 to 4 were mostly designed and 
executed in similar ways under the direction and 
facilitation of Initiative leaders. Beginning in 
Year 5, as Initiative grant funding was reduced, 
each district adapted the TLC model to best meet 


its circumstances and goals. 
This report addresses the following questions: 
> What are the benefits and challenges of TLCs? 


> What was the Initiative-wide configuration of 
TLCs in Years 1 to 4? 


> How did Early Implementer districts modify 
the original lesson study configuration in 
Year 5? 


1 The Summer Institutes were weeklong professional learning events that kicked off each year of the Initiative. (See Glossary 
for a full definition of this and other specialized terms associated with the Initiative.) 


2 All teachers participating in the Early Implementers Initiative received professional learning in leadership and were referred to 


as Teacher Leaders. (See Glossary for a full definition of this and other specialized terms associated with the Initiative.) 
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> What gains and losses have participants 
attributed to the different lesson study adap- 
tations in Year 5, compared to the Year 1to 4 
model? 


> What are the most important ways adminis- 
trators can support implementation of lesson 
studies? 


Like the entire evaluation series for the NGSS 
Early Implementers Initiative, this report aims to 
be useful to school and district administrators, 
leaders of science professional learning, and state 
policymakers by sharing the Initiative's learnings 
in understandable and replicable ways — and 
through that, to help inform those leaders’ own 
policy, practice, and resource-allocation decisions. 
As described in Appendix A, this report's findings 
are based on an extensive amount of data: evalu- 
ators’ observation of 27 TLCs; data from a dozen 
surveys; and over 100 interviews with teachers, 
administrators, district Project Directors, and 


K-12 Alliance Regional Directors. 


The Teaching Learning 
Collaborative: The Early 
Implementer Lesson Study 


What is the Initiative's Teaching Learning 
Collaborative and where did the model come from? 
By the time staff from WestEd's K-12 Alliance 
started planning the Early Implementers Initiative 
in 2012, they had spent almost 20 years facilitat- 
ing their own type of lesson study with teachers 

of science. Beginning in 1995, when they discov- 
ered that many teachers were not implementing 
changes they had learned during summer profes- 
sional learning, the K-12 Alliance team developed 


a classroom-based activity — informed by what 
they knew was successful in coaching models — 
to reinforce participants’ summer professional 
learning. They called the strategy the Teaching 
Learning Collaborative, or TLC. This model partic- 
ularly emphasizes the importance and value of 
building communities of practice around the 
improvement of teaching and learning (Lave & 
Wenger, 2001). 


TLCs were a cornerstone professional learning 
activity for all Teacher Leaders participating 

in the first four years of the Initiative (Tyler & 
DiRanna, 2018). Beginning in Year 5, when profes- 
sional learning became less centralized, districts 
implemented modifications of TLCs that will be 
described later in this report. 


In the Initiative’s TLCs, a team typically comprised 
three to four grade-specific teachers from differ- 
ent schools in the district who met with a trained 
facilitator for two full school days. The first day is 
spent planning a lesson that will be taught by the 
team one to three weeks later, on the second day 


they meet. 


> Day one: The lesson planning session begins 
with a brainstorm of what a student would 
need to understand in order to be able to 
explain a chosen phenomenon. The team then 
creates a “storyline” (aka "conceptual flow") 
showing the flow of concepts in an order that 
would make sense for students to learn them. 
From there, the group zeros in on a chunk 
of the storyline that will be the focus of the 
lesson. Using a template based on the 5E 
instructional model,? the team records specific 
teacher actions along with expected student 
responses and target concepts for each "E" 
stage (i.e., Engage, Explore, Explain, Elaborate/ 
Extend, Evaluate). 


3  5E is an instructional model that is valuable for NGSS instruction because it prompts the instructor(s) to plan a student- 
centered, inquiry-based lesson based on how people learn (National Research Council, 2000). See Appendix B for a brief 
explanation of the 5E instructional model. For a more detailed description, see special evaluation report Next Generation 
Science Standards in Practice: Tools and Processes Used by the California NGSS Early Implementers: https://www.wested.org/ 


resources/next-generation-science-standards-in-practice/ 
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> Day two: In the teaching session, participants 
co-teach the lesson twice with two different 
classes or groups of students while the facil- 
itator records student and teacher actions. 
Between the teachings, the team considers 
evidence, including the facilitator notes and 
student work generated during the lesson 
(Darling-Hammond, 1998), as they revise the 
lesson for the second teaching. The teaching 
day concludes with a final debrief where 
evidence from the second lesson is discussed 
to evaluate the effectiveness of the revisions 
made to the first lesson. Participants reflect 
on what they learned from the experience and 
how they might use the lesson in their regular 
instruction. 


Teachers often find TLCs to be a powerful profes- 
sional learning strategy because they are: 


> Facilitated: Trained leaders help the teams 
progress through the steps. The careful notes 
they take during the lesson provide useful 
evidence when evaluating the effectiveness of 
the lesson. 


> Collaborative: Teachers find it easier to take 
risks in a safe space with others, and they 
find value in hearing one another's questions 
and ideas. This is particularly the case in the 
debrief, during which teachers review student 
work and use that evidence to evaluate the 
lesson. 


> Based ina real classroom: Unlike most teacher 
professional learning, TLCs allow teachers to 
experience what it looks and feels like to prac- 
tice new pedagogy in the classroom. 


Most Critical Supports 
from Administrators 


Early Implementer leaders discovered even at the 
outset of the Initiative in Year 1 just how import- 
ant principal and central office administrator 


support was to successful NGSS implementation. 
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"AS an administrator, I’m there to encourage 
teachers as they are taking new steps forward.” 


[Elementary school principal 


As TLCs require teachers to be out of their class- 
rooms for two full school days, the TLCs could not 
have taken place without critical supports from 
site and district administrators. In particular, 
administrators provided the following commit- 
ments and resources to make the TLCs a success- 


ful cornerstone of the Initiative. 


Making science a core subject. When applying to 
join the Initiative, districts agreed to make science 
a core subject. Several districts established a mini- 
mum number of minutes of science instruction per 
week in elementary schools and prioritized science 
in districtwide professional learning. All districts 
allowed participating teachers to be released 

from class for four TLC days per year and made 
arrangements for substitute teachers available to 


cover their classes during that time. 


Providing release time and funding. In order to 
participate in two TLCs each school year, teachers 
had to miss four full days with their students. 
While the Initiative grant provided funding to 
cover the costs of teachers participating in the 
TLCs, districts still had to find ways to execute 
release time and arrange and pay for the substi- 
tutes. Project Directors worked to participate 

in Local Control and Accountability Plan (LCAP) 
meetings, with those in smaller districts often 
having better success. In districts where they 
succeeded, Early Implementers enjoyed reliable 
funding for substitute teachers as well as supplies 
for NGSS lessons. For the most part, all eight 
districts did what was needed for Teacher Leaders 
to participate in their two TLCs during each of 
the first four years of the Initiative. Nonetheless, 


Project Directors in some of the districts where 
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NGSS implementation was not included in the —_— IE 


EGA) found theniselves newng co ean inaaly “When | attended the TLC] | just sat with my teachers. 
make their case for district resources, especially 


We had already started the school year and we had 
substitute teachers. 
already told them that they needed to have X amount 


Support from principals. In order to partici- of minutes per week of science. So, they were a little 


pate in TLCs, teachers must have the active and apprehensive in the beginning. But [after the TLC], they 
consistent approval of their principals. This was 


felt more comfortable with [teaching the NGSS], and 
relatively easy to obtain when arrangements 
So | felt like | was able to help support them and have 


their back in that process.” [K-8 principal] 


were negotiated months in advance. However, it 
sometimes took dedicated principal support to 
help protect a teacher's TLC attendance in the 
face of a conflict or last-minute issue. Although of teachers’ participation in TLCs as the principals’ 
uncommon, it was not unheard of forateacherto | understanding of the NGSS and the shifts required 
be called back to school after a TLC day had begun _to teach them grew. By Year 3 of the Initiative, 
because his or her substitute teacher was needed many site administrators made time to attend 


for a “more important” purpose. For most teach- TLC planning and teaching days themselves — 
ers in this circumstance, there was no make-up even, although less often, at schools other than 
TLC session. their own. Even just the principals’ presence at 


a TLC sent a strong message of approval and 


In interviews, teachers and administrators encouragement to teachers to participate in TLCs 


reported that principals became more supportive 


and to transition to NGSS instruction. 
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A Vehicle for 
Teacher Motivation 


To begin to implement the NGSS, most teachers 


must make major changes in their teaching. TLCs 
provide a safe space for teachers to try out new 
NGSS instructional practices as a member of a 
small, supportive team experiencing similar radi- 
cal shifts in their practice. For elementary teach- 
ers, lesson studies provide an additional invaluable 
opportunity to learn about teaching science and 
to begin to understand that they need not have 
mastery of all the scientific content. 


In surveys, Teacher Leaders at all grade levels 
reported feeling more confident about teaching 
the NGSS after spending two days with their TLC 
team and TLC facilitator. By having an active 

role in the planning, teaching, and evaluation of 

a three-dimensional NGSS lesson, participants 
realize that they can successfully adopt new NGSS 
teaching practices. 


Teachers were apprehensive about the 
change in science teaching and required 
[minimum science instruction] minutes 
but time with other teachers showed 
them it was possible and that they had a 
community to support them. (Elementary 


school principal) 


I like the opportunity it provided for me to 
collaborate and share/modify/try [NGSS] 


lessons. This experience really helps me 


feel more comfortable with the standards, 
the process of planning, and the delivery of 


a 5E lesson sequence. (Third grade teacher) 


I have increased my knowledge about 
lesson planning from our TLCs. I have 
learned how to revise and make our lesson 
more explicit both for the teacher as well 
as the student outcomes. It was a great 
way to collaborate, disagree with some 
things, and eventually come to an agree- 
ment. All this, done in a safe and welcom- 
ing environment, was amazing. (First 


grade teacher) 


For middle school teachers, the NGSS have reallo- 
cated content across grades 6, 7, and 8, making it 
necessary for some to brush up on concepts they 
have not taught recently, or ever. The group TLC 
process can help these teachers get on board with 
this restructured content, as this middle school 


principal explains: 


I've seen a real focus in teachers wanting 
to plan lessons. [They're] not saying, “Oh, 
yeah, I used to do this thing before NGSS 
so I'm still gonna do this thing and I’m 
gonna call it NGSS.”... They're under- 
standing how things have to change and 
they're excited about it. They want to 
create new units and new lessons. (Middle 


school principal) 
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Table 1. Percentage of survey takers who chose answer choices “Moderately” or “A lot” for 
survey question, “To what extent did your TLC experiences in this SCHOOL YEAR deepen 
your understanding of the following aspects of the NGSS?” 


rAX-J ol Tol Mo) dal LEN) 


Science Disciplinary Core Ideas (DCIs) 


How to use a three-dimensional 
approach to help students understand a 
phenomenon 


How to use Science and Engineering 
Practices (SEPs) to teach science and 
engineering 


How to use the Crosscutting Concepts 
(CCCs) to teach science and engineering 


How to use the engineering 
design process to develop student 
understanding of science and 
engineering 


How to integrate the science 
disciplines (e.g., physical, earth & space, 
life) to increase student learning 


Source: Retrospective TLC Survey administered to all Teacher Leaders by WestEd in spring 2016 (N=353) and spring 2017 (N=388). 


Note: Other answer choices were “Not at all” and “A little.” 


Understanding the 
Standards 


In surveys administered at the end of their first 
and second years of participation in the Initiative, 
teachers were asked how much their TLC expe- 
riences had deepened their understanding of 

the three dimensions (i.e., science disciplinary 
core ideas, science and engineering practices, 
and crosscutting concepts) and other aspects of 
NGSS. As shown in Table 1, the high percentages 


Percentage choos- 


Tate Mm Utete(=1e-1¢-1\aame) g 
Na Co) ails 40) Bo Ea Ko) 


1 c=] a t= 10 [= Mel aloloy- 


Tate Mm utete(-1e- 1c aol g 
“A lot” in 2016-17 


84% 84% 
78% 78% 
76% 79% 
78% 81% 
58% 63% 
69% 72% 


of teachers who reported their understanding 
was deepened “moderately” or “a lot” by their 
TLC experience stayed relatively consistent 
across the two years for all of the NGSS aspects 
(69-84 percent except for engineering design). 


A likely reason that teachers said they gained less 
understanding about “How to use the engineering 
design process to develop student understand- 
ing of science and engineering” than other NGSS 
aspects is that engineering was not a component 
of many TLCs in the early years of the Initiative.* 


4 Because the NGSS require many major shifts, the Initiative decided in early years to place an emphasis on most but not all 
aspects of the NGSS for elementary-level teachers to avoid overwhelming and deterring teachers’ interest. 
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Table 2. Percentage of survey takers who chose answer choices “Moderately” or “A lot” for 
survey question, “To what extent did your TLC experiences empower you to be able to use 
the following project tools and practices on your own (should you wish to use them)?” 


Project tool 


How to use the 5E instructional model 
to design and teach lessons 


How to use phenomena to drive 
instruction 


How to use questioning strategies 
(e.g., teacher-to-student, student-to- 
student discourse) to develop student 
understanding 


How to look deeply and systematically 
at student work as evidence of student 
understanding 


How to use science notebooks to elicit 
student sense-making 


Percentage choos- 


"A lot" in 2015-16 


Percentage choos- 


Take mm utete(-1e-1¢-1\amol am Mlale mm iy Cole(-1¢-1<-1\\ ae) g 


"A lot” in 2016-17 


84% 84% 
84% 87% 
83% 86% 
83% 85% 
76% 79% 


Sources: Retrospective TLC Survey administered to all Teacher Leaders by WestEd in spring 2016 (N=353) and spring 2017 (N=388). 


Advancing NGSS Teaching 


Teachers also rated the TLC experience highly 
when asked about its value in preparing them to 
adopt new teaching practices. Teachers were asked 
how much they felt their TLC experience empow- 
ered them to use the five main tools and processes 
they had learned in the TLCs: the 5E instructional 
model, phenomena, questioning strategies, review- 
ing student work, and notebooks. Teachers had 
been introduced to all of these tools and processes 
to different degrees during the Summer Institutes; 
TLCs demonstrated their use in a classroom 
setting. Across two school years, over 80 percent 
of respondents reported that their TLC experience 
had empowered their use of four out of the five 


tools and processes either “moderately” or “a lot,” 
with over three quarters (76 percent) reporting 
that their TLC experience had empowered their 
use of the fifth tool, notebooks, “moderately” or “a 
lot” (as shown in Table 2). Interestingly, of these five 
tools, science notebooks was the tool that teach- 
ers learned the most about outside of the TLCs, as 
there were sessions devoted to them during each 
Summer Institute beginning in Year 2. This may 
account for the slightly lower ratings, since teach- 
ers already felt empowered to use notebooks. 


Beyond these tools and processes, in surveys taken 
at the completion of each TLC during Year 2 and 
Year 3 of the Initiative (2016-17 and 2017-18), teach- 
ers were asked, “Please clearly describe one thing 
you learned from this TLC experience that you 


5 See Appendix B for an explanation of these five project tools and practices. 
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plan to take back to your classroom.” Responses teaching practices. The effectiveness of 
provide evidence that teachers not only under- 
stood the intent of the three-dimensional NGSS, 


but also that they had experienced the power of 


asking questions of the students instead 
of telling them what you want them to 
know was explicitly clear during our 


implementing the standards with students: ; } 
lesson. When we practiced the lesson in 


I love the questioning we did and the 
pushing back we did to get the students to 
think. We pushed them to look for evidence 
of their thinking rather than guiding them 
to a specific answer. I plan to use this 
tomorrow in my own classroom. I want 
the kids to think for themselves and not 
always search for the “right answer." 
(Third grade teacher) 


the classroom, I was surprised at how 
much students were really able to connect 
their thinking and draw their own conclu- 
sions based on the evidence they observed 
in the investigation, without any direct 
instruction from the teacher. I noticed 
that the “teacher” role has transformed 
from information giver to facilitator of 


thinking processes. (First grade teacher) 


We changed the assessment piece at the The TLCs provided teachers with the opportunity 


end of the lesson for the second time we to experiment and evaluate new instructional 


taught it. It was interesting to see how approaches with peers. The process of analyzing 


much more the child was able to explain how well a lesson actually progressed student 


and think about with the second. more understanding changed many teachers’ perspec- 
tive, as explained during interviews by these two 
open-ended assessment worksheet. I ; ie y = 

Project Directors: 


know that I will be able to find out more 
about what my students know if they are 
given the chance to answer more open- 
ended questions that are there to uncover 


their thinking. (Kindergarten teacher) 


I was impressed that some of my lowest 
students in math and reading are my 
top students that make the big connec- 
tions. I also learned how to constructively 
critique student work so that I am able to 
move my kids to the next level. (Second 


grade teacher) 


This being my first TLC, I learned the 


power of inquiry-based learning and 


I think that the best thing that I've seen 
in a TLC is that teachers are starting to 
look at what didn't work. They're starting 
to think more about what the students are 


doing than just how to teach [the content]. 


When teachers come to their very first 
lesson study ever, it can blow their mind. 
It's really hard for them sometimes to think 
about spending six and a half or seven 
hours to basically plan one lesson. They 
start to see how complex the NGSS really 
are and how much thought should go into 
doing it well. So, what's made me really, 


really happy is when I have a team... If 
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I'm facilitating, they're almost all teachers 
brand new to the process, and their under- 
standing is very fragile. And so, for them 
to sit here and plan a really thoughtful, 
complex lesson and for them to not under- 


estimate their kids is amazing. 


Over time, some teachers saw the value of having 
lesson studies in other subject areas and in apply- 
ing lessons learned to other subjects, as articu- 
lated by the following interviewees: 


There was so much positive feedback 
from the [TLC] experience that they 
now want to do that experience in other 
subject areas as well — math and ELA. 


(Elementary school principal) 


The one thing that I have learned from this 
TLC experience is that I can take a lot of 
the teaching strategies back to my class- 
room and use them across the curriculum. 
For instance, to not tell the student that 
their answer is wrong in math but have 
them discover why it is wrong by asking 
probing questions that will help them 
discover a new way to solve the problem. 


(First grade teacher) 


Benefits for Administrators 


The Initiative has made a point of leveraging TLCs 
to help administrators understand the NGSS so 
that they may better support their teachers.® 


When asked in a survey in spring of 2018 about the 
extent to which observing various experiences had 
impacted their understanding of the NGSS, prin- 
cipals reported that they found benefits in observ- 
ing both days of the TLC, although the teaching 
day was more meaningful for them than the lesson 
planning day, as shown in Figure 1 on page 10. 


In interviews, principals gave more specifics on 
what they had gained from their involvement in 
TLCs, for example: 


I have learned a lot from the TLCs, like 
how to build a conceptual flow... the 5E 
structure ... notebooking and how that 
works. I feel like, as an admin, I have a 
strong knowledge of what the teachers 
do to implement the NGSS and how it’s a 
new way of approaching teaching science. 


(Elementary school principal) 


Of course, what principals generally care most 
about is how their teachers are doing, and their 
responses were very positive about what their 


teachers were learning. 


I think there are many principals who 
are very happy with the Initiative. I think 
they're confident with teachers who are in 
the Initiative. [They're] happy that teach- 
ers are receiving a higher level of PD and 
collaboration within it. (Middle school 


principal) 


6 Principals in Early Implementer districts have been strongly encouraged to observe teachers engage in the TLC process. 
Seeing the TLC lesson and the post-lesson debrief helps administrators understand the complexity of the NGSS and the shifts 
in pedagogy required to teach them. The crucial role of administrators in NGSS implementation is the focus of completed 
evaluation report #3, Administrators Matter in NGSS Implementation: How School and District Leaders Are Making Science 
Happen, available here: https://www.wested.org/resources/administrators-matter-ngss-implementation/. This topic will be 


revisited in upcoming evaluation report #10, slated for release in fall 2019. 
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Figure 1. How principals rated the level of impact of observing TLC planning and teaching 
days on their understanding of the NGSS 


Principals: During the 2017-18 school year, how much of an impact did the following have on your 
understanding of NGSS? 


100% 
90% 
80% 
70% 
60% 
50% 
40% 
30% 
20% 
10% 

0% 


Observing TLC lesson planning day Observing TLC teaching day 


@ None @ Alittle @ Some’ ®©& Agreat deal 


Source: Principal Survey administered by WestEd in spring 2018 (N=77). 


Teaching science is about building their up at because it was pretty phenomenal. 
capacity as a teacher. I think that learning (Elementary school principal) 


how to [plan] an effective science lesson 


is just also helping you become a better Bringing groups of teachers together from differ- 
teacher. So, I think it’s hugely impactful ent schools to engage in professional learning 
because it goes past just teaching science. based on collaboration and team-building for four 
(Elementary school principal) years has inspired changes in the values and prac- 


tice of not only the teachers directly participating 
in the TLCs, but their schools and districts as well. 


Im pact on Site and District The Imitiative's TLCs reflect a “growth mindset” 


Ss U ltu re (Dweck, 2006) and model what teachers should do 
in their own classrooms. As described by an Early 
It's just . .. so supportive to watch teach- Implementer Core Teacher Leader in a California 


ers learn from each other and support Science Teachers Association (CSTA) newslet- 


- E ; ter article, the learning process of students in 
each other. I'd like to continue that 
an NGSS classroom parallels that of the teacher 


method no matter what school site I end engaged in a TLC Gallagher, 2018). For both 
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students and teachers, the experience prioritizes 
inquiry-based activities that allow each individ- 
ual to jump in where they are and develop deeper 
or new understanding through exploration and 
discourse. The main goal of participating in the 
TLCs is not to determine what is “correct” or 
“incorrect.” Instead, everyone is simply on the 
path from their original understanding (which 
may be naive, incorrect, or incomplete) to form- 
ing a conceptual framework that makes the most 
sense based on evidence. The more experience and 
investigation one engages in, the better one can 
form concepts that are consistent with evidence 
(DiRanna, Topps, Cerwin, & Gomez-Zwiep, 2009; 
Tyler & DiRanna, 2018). Through involving more 
teachers and administrators over time, the 
growth mindset culture of the Early Implementers 
Initiative has begun to permeate not only class- 
rooms, but schools and districts as well: 


They [the teachers participating in TLCs] 
are very collaborative. If they don't know 
an answer, they don’t shut down. They 
know who to go to, they take risks. If 
they're not sure about what to do with 
their teaching, they tend to check in with 
each other and work on something, they 
don't shut down. So let's just say that 
they have a growth mindset, and that’s a 
result of the Initiative. (Elementary school 


principal) 


I think, when given the time [to collabo- 
rate], that they just have amazing ideas 
and plans. They can have this lesson and 
they can all share in teaching it. And then 
when they come back together, that’s 
the very cool time. You know, they come 
back together and they're like, “Well, did 
you notice this? And I changed this...” 


... I thought it was just fascinating how 
everyone is valued in what they're saying. 
And they can just share and there's none 
of the judgments on anyone. I was real 
pleased with the openness of people. And 
I think that’s because we've been able 
to do this for a couple of years now ... 
they're very comfortable with the process. 


(Elementary school principal) 


The collaborative culture extends throughout the 
district. TLCs provide a context in which admin- 
istrators can listen to teachers to learn about the 
standards and gain insight into the complexity 

of planning NGSS instruction. The focus is on 
learning as a process rather than one specific 
outcome. As they learn through their involve- 
ment with TLCs, administrators improve their 
understanding of how to support their teachers in 
teaching the NGSS, which further impacts district 
and school culture. District administrators share 
a common goal with teachers and principals: to 
support students in developing learning skills 
(e.g., inquiry, communication, metacognition) to 
construct a conceptual framework for making 


sense of phenomena and related concepts. 


One of the biggest hurdles for the last four 
years is getting teachers to not worry 
about administrators coming in and judg- 
ing them because they're at the beginning 
of a sequence, and kids have no back- 
ground [so they may have misconcep- 
tions]. The teachers are kind of feeling out 
the kids for what they do and don't know. 
Things are gonna get tweaked [to meet the 
students’ needs] so that they will under- 
stand the concept that’s being taught... . 


The administrators understand that’s 
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what they're gonna see and it’s okay. 


(Middle school principal) 


[My role at the TLC] is really just to 
observe. . . love when [the teachers] make 
a connection where they turn and ask me 
to help support with building a community 
partner, or bringing in an expert guest 
speaker.... They know that's something 
I can support with. But in terms of the 
actual lesson planning and instructional 
implementation, that's really ... our 
Project Director leading the teachers, and 
the Teacher Leaders leading the teachers. 
As an admin, I just kind of sit and listen 
and wait for ways that I can help and 


support. (Elementary school principal) 


It's funny, my boss, who comes and does 
walkthroughs in my classrooms, told me, 
“I love coming to your school because 


when I come here, I see joy on the kids’ 


faces, joy on the teachers’ faces.” And 


she told me if she had to do it over again, 
she'd put her son in my school. It’s just 
because [we have] this mindset that the 
kids need to be engaged and be thinking. 


(Elementary school principal) 


Challenges 


While feedback on the TLCs was overwhelmingly 
positive, there were also challenges. The most 
common challenges were logistical. District 
Project Directors were faced with the complex 
task of assigning dozens of teachers to teams of 
three or four members each; finding times and 
locations for each team to meet (twice in the fall 
and twice in the spring); and procuring and paying 
for substitute teachers so that teachers could be 
released from class for their TLCs. Some adminis- 
trators were less than enthusiastic about teachers 
being out of the classroom. 


Many of these challenges inspired modifications to 
the TLCs in Year 5, described in a later section of 
this report. 
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As noted in the report's introduction, each TLC 


in Years 1 to 4 consisted of two days: a lesson 
planning day and a teaching day. Every Teacher 
Leader experienced two TLCs per year, one in the 
fall, and one in the spring, and individuals stayed 
in their grade-specific teams for the school year. 
TLCs in these first four years were facilitated by 
the Initiative’s leadership, the K-12 Alliance, the 
district Project Directors, and the Core Teacher 
Leaders. Obviously, the TLCs require quite a time 
commitment — four full days over the course 

of the school year — and some teachers initially 
balked at the notion of spending two days that 
ultimately result in just one lesson. However, as 
observed by evaluators and described by a district 
Project Director in a California Science Teachers 
Association (CSTA) newsletter article, teach- 

ers quickly saw the value of the experience (as 
described in the Benefits of the Early Implementer 
TLCs section above) and, from there, moved to 
deeper professional concerns, such as how to 
continue and even expand application of the TLCs 
(Tupper, 2018). 


Vignette: An Extended 
Example of a TLC in Year 2 


To provide readers with a more vivid understand- 
ing of what TLCs are like in practice, the following 
vignette describes a grade 6 TLC from Year 2 that 
is representative of many of the TLCs that Early 
Implementer evaluators observed.’ This TLC 


convened four teachers and a principal, represent- 
ing five different schools, along with the district 
Project Director and the district’s assigned Early 
Implementer Regional Director from the K-12 
Alliance. The teachers were a mix of Core Teacher 
Leaders (in their second year of the Initiative) and 
Teacher Leaders (in their first year). The principal, 
a member of the Core Leadership Team, was not 
able to participate in the TLC consistently but 


acted as an occasional advisor. 


Planning Day 


The Regional Director introduced the lesson study 
process, beginning with a discussion on teacher 
participation norms for collaboration, discussion, 
and other processes. When the group moved to 
discussing the lesson to be taught, Teacher A, 
whose class they would be teaching, said she 
wanted to do the lesson on something related to 
cells, because it would fit with what the class was 
currently learning. After at least an hour of open 
discussion on potential specific lesson ideas, the 
Regional Director instructed everyone to close 
their books and think about what all students in 
grade 6 should know about cells by the end of the 
school year. Individually they made lists. They 
shared out ideas, creating a storyline, or “concep- 
tual flow,” of the instructional unit. The big ideas, 
in the order that students could learn them, were: 


i. Living and nonliving things are different; 
living organisms have certain characteristics. 


7 For another vignette of an Early Implementer lesson study, this one in Year 4 of the Initiative, see the article, “"NGSS and the 


Teaching Learning Collaborative: It’s About the Process” (Tupper, 2018) in California Classroom Science by an Early Implementer 
Project Director: http://www.classroomscience.org/ngss-and-the-teaching-learning-collaborative-its-about-the-process. 
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2. Living things are made of cells; cells are made 
up of parts that each have a function and 
structure; single-cell and multicellular organ- 
isms are different; not all cells are the same in 
a multi-cell organism. 


3. Cells become organs and tissues; body systems 
are made of cells; some cells are specialized 
cells; there are differences between plant 
versus animal cells. 


Teacher A had already started teaching about 
cell structure, and the process of creating the 
conceptual flow caused her to realize that she 
really should have covered the difference between 
living and nonliving things before introducing 
cells. The group decided that this would be an 
appropriate concept for a "5E” TLC lesson, which 
takes students through a series of learning expe- 
riences based on how people learn, also known as 
the five "Es": Engage, Explore, Explain, Extend, 
and Evaluate. 


The lesson would focus on the differences between 
living and nonliving things, with the goal of iden- 
tifying the traits that all living things have and 
laying the groundwork for lesson(s) about cells. 

It would also set the stage for further learning, 
which would involve reading more information 


and revising the list of characteristics. 


As the group discussed characteristics of living 
things that grade 6 students should learn, they 
considered potential phenomena to drive the 
lesson, which they would introduce in the very 
first part of the lesson ("Engage"). They decided 
to highlight one characteristic of living things, 
inheritance of traits, by kicking off the lesson 
with a comparison of a large and a small tree of 
the same species. Students would share prior 
knowledge of what distinguishes a living from 
anonliving thing, such as inheriting traits from 
parents, and then engage in a sorting activity to 


further explore this concept. 


One person asked about volcanoes: Would 
students think volcanoes were alive? The group 
brainstormed some obvious examples of living 
versus nonliving things, and then some less obvi- 
ous examples. They decided to have students sort 
pictures of living and nonliving things for the first 
“Explore” part of the lesson. 


When someone brought up the idea of using 
microscopes, the principal said she could make 
some available, but Teacher A said her students 
wouldn't know how to use them. They decided to 
use two sets of pictures or, if needed, videos for 
students to sort. One set would be more obvious, 
while the other would include some trickier things 
for students to identify as “living” or “nonliving." 
They brainstormed ideas. For the first set of 
pictures or videos to sort: ivy, pencils, a snake, a 
water bottle, a fly, a frog, a shoe, a basketball, a 
log, arock, a mushroom, a shark, a tree, children. 
For the second set to sort: an anemone, a car, a 
feather, fire, a video of crystals that grow, a video 
of a volcano, a dinosaur, bacteria, driftwood, 

sun, water. They created a graphic organizer for 
students to record which items were living and 
which were not, as well as traits of living and 
nonliving things. This would be the student work 
collected at the end of the lesson. 


After lunch, they started recording in the Lesson 
Plan template what the Teacher Does, what the 


Student Does, and what Concepts are covered. 


The template they used prompted them to follow 
the 5Es, and their single-period lesson plan got 
through the first three (Engage, Explore, Explain), 
with three iterations of Explore and Explain in 
which students would use new information and 
ideas to develop and revise their explanations. 
While the group continued to plan, one teacher 
wrote in the template, while two others looked 
for an appropriate reading passage, and another 
teacher looked online for photos and videos to use 
in the lesson. They listed the characteristics of all 
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living things that they wanted students to know; 


the reading passage would need to include them all. 


By the end of the planning day, they had identified 
the most relevant NGSS standard as well as the 
most relevant disciplinary content knowledge 
(DCI), science and engineering practices (SEPs), 
and cross-cutting concepts (CCCs). They had a 
lesson planned from start to finish, a reading 
passage that needed a bit of tweaking, and most 
of the photos/videos. The Regional Director kept 
track of who would prepare the reading passage, 
who would cut and laminate the cards, who would 
make copies of the graphic organizer, and who 


would put the videos into a PowerPoint. 


Teaching Day 


Two weeks later, the full group, including the 
principal, reconvened early in the morning for 
their teaching day. A reading passage had been 
prepared on "Living Versus Nonliving Things," 

the 5E template had been fleshed out and copies 
were provided for each participant by the Project 
Director, and a handout for students had been 
created. The team discussed who would teach each 
section of the lesson and how long each section 
should take. 


First Teaching Session. Teacher A had a class of 
32 students, which included an additional five that 
join the class for science. Students were seated 

in eight groups of four, and it was clear that they 
had plenty of practice working cooperatively in 
small groups. The first teacher led the class as 
they talked about what makes a tree get bigger. 
Then, a second teacher took the lead, and the 
class sorted the first round of pictures and videos 
in their groups, recording ideas in their graphic 
organizers. They shared as a whole class about 
characteristics of living versus nonliving things, 
with another teacher recording their ideas on 
the board. A third teacher led the next activity. 


Students were given a new set of pictures and they 


watched videos of the crystals, the volcano, and 


the fire. The second sorting was a bit more diffi- 


cult, and many groups incorrectly categorized at 


least one nonliving thing as a living thing, most 


typically the sun, the volcano, and/or the fire. Time 


was spent at the end of the class, led by the fourth 


teacher, on revisiting the list of characteristics of 


living things they had brainstormed after the first 


sorting activity. The students were given time to 


revise their graphic organizers. They would read 


the reading passage the following day and make 


further revisions to the list of characteristics. 


The Debrief. The group discussed what they 


thought were more effective and less effective parts 


of the lesson and then read some student work. The 


work was distributed and teachers created "high," 


“medium,” and “low” piles, reading aloud to each 


other student work that was or was not on track. 


They counted totals for each performance level 


and discussed features of the lesson that seemed to 


help and hinder student understanding. 


The group felt that some parts of the lesson had 


worked pretty well: 


> 


Vv 


The graphic organizer kept students’ focus 
and there were no questions about how to use 
it. They agreed that if there isn't an individual 
task, sometimes kids get off task. 


Sorting through the images of living and 
nonliving things was the most powerful thing 
students did. It was helpful for them to phys- 
ically manipulate the pictures. The teachers 
could walk around and see where misconcep- 
tions were. 


Timing was good. Students stayed on task and 
kept moving through the task. One teacher 
noted, "Sometimes people say you can't do 
science if you don't have a huge chunk of time 
but look what we did in an hour!" 


The videos worked very well. 
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> The second sort elicited a lot of discussion. 
Team members noted: “Items in the second 
sort were great!” “The dinosaur in the picture 
was plastic, and maybe that was distracting.” 
“Some were distracted, or debated [with each 
other], because something was no longer alive." 
“I heard a group saying that the sun needed 
energy.” "I heard cells come up a lot.” "Some 
students said everything has cells." 


> It worked to have two rounds of sorting living 
and nonliving things. The first sort activity got 
students thinking, and that helped them do the 
second sort. 


> The Regional Director noted that it was possi- 
ble to see the influence of the Common Core 
State Standards in students’ argumentation; 
they are getting more used to saying their 
thinking. Teacher A recalled that one of her 
students noted aloud that she really didn't 
know what gravity is: “It’s really cool that 
we're getting kids to say what their thinking 
is, because I thought I knew, but I had no idea. 
And now kids are not only showing us what 
their deep understanding is, they are realizing 
what they themselves don't understand.” 


The group also recommended some changes to the 


lesson: 


> Give students more room to write. They had 
to write full sentences and providing only two 
lines on the graphic organizer was limiting. 
One group member noted, “I saw one girl who 
had another idea. Why should I write that 
down? I'm done, my lines are full.” 


> Maybe the trees made students focus too much 
on plants. The group wondered whether they 
should include both plants and animals. 


The next class period in which the group would 
teach this lesson was shorter. So, they decided that 
the initial Engage (looking at the small and large 
trees and thinking about how the tree got bigger) 


could be cut, because it didn't really inform the 


main part of the lesson or the sorting activity. 


Second Teaching Session. This class had 

20 students, seated in five groups of four. The 
lesson was taught as before with the agreed-upon 
changes. The students were not as practiced 
with small-group cooperation, but the ideas they 
shared about characteristics of living things 
were on target. The second round of sorting 
elicited disagreements among students, some 
very heated. There was time for a final review 
and re-think of what the characteristics of living 
things were. The class ended with one teacher 
saying, “Maybe we need more information, maybe 


we should do some reading.” 


Second Debrief. The group talked about how good 
it was to emphasize to students that scientists 
classify and categorize. They noted that it might 
be necessary to reword the directions, as students 
were writing one characteristic for living things 
(e.g., they inherit traits) and then the opposite for 
nonliving (e.g., they do not inherit traits). They 
closed with an individual quick write on what they 
had learned over the course of the two teaching 
sessions and what they planned to take back to 


their own classrooms. 


Developing Facilitators 
for More Widespread 
Implementation 


As teachers learn the standards and advance their 
ability to teach them, they also become more will- 
ing and able to share what they know with their 
peers, which was an explicit expectation for all 


Teacher Leaders in the Initiative.® 


8 See previous evaluation report on teacher leadership, Investing in Science Teacher Leadership: Strategies and Impacts of 
the NGSS Early Implementers Initiative: https://wested.org/resources/investing-in-science-teacher-leadership-ngss-early- 


implementers/ 
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Table 3. Who was the primary facilitator of your TLC? 


Primary facilitator of TLC | Fall Nye} dials} Fall NY ol diate} 
2015-16 2015-16 2016-17 2016-17 
Regional Director 36% 28% 21% 17% 
Project Director 40% 38% 37% 38% 
Core Teacher Leader 25% 34% 38% 40% 
Other NA NA 8% 4% 


Sources: Post-TLC Survey taken by participants at the conclusion of their TLCs during the 2015-16 and 2016-17 school years 
(N=369 in fall 2015, 356 in spring 2016, 353 in fall 2016, 368 in spring 2017). 


Notes: Respondents chose “Other” when the TLC was facilitated by someone other than the three leaders shown, or the 
facilitator’s role in the Initiative was unknown. “Other” was not an answer choice in 2015-16. 


I see that confidence has grown, not only 
in understanding content, but also in 
being a Teacher Leader and presenting 
to each other, facilitating, and taking the 
initiative. ... They had to present to each 
other, and they had to present at some 
of our ... district professional learning 
community days where they present to 
everyone, to all of their peers, so that’s 


been great. (Elementary school principal) 


I plan to continue to plan a scope and 
sequence of lessons using the concep- 
tual flow plans that our TLC teams have 
created. I also hope to encourage, mentor, 
and support my grade-level team in 
teaching more science within their own 


classrooms. (First grade teacher) 


five administrators per district) who each engaged 
in two TLCs that year. Groups of 30 to 70 Teacher 
Leaders in each district joined in Year 2, and they 
too participated in a TLC in the fall and another in 
the spring. By Years 3 and 4, some of the Teacher 
Leaders began to actively share what they were 
learning about the NGSS with teachers outside of 
the Teacher Leader groups.? However, the TLCs 
were reserved only for direct participants of the 


Initiative. 


As Core Teacher Leaders and Teacher Leaders 
gained expertise in the NGSS and experience with 
TLCs, the number of individuals in each district 
able to facilitate increased. Leadership train- 

ings held twice a year for Core Leadership Team 
members regularly included sessions to help Core 
Teacher Leaders improve their TLC facilitating 
skills. In Year 1, all of the facilitators were Regional 
Directors from the K-12 Alliance, but by the spring 
Project Directors and some Core Teacher Leaders 
were “shadowing” in preparation to facilitate the 


In Year 1 of the Initiative, TLC participants following year. Table 3 illustrates how responsi- 


included only the district Project Directors and bility for facilitating TLCs gradually transferred 
the members of the Core Leadership Teams from the Regional Director to the Project Director, 


(approximately six to eight teachers and two to and then to the Core Teacher Leaders. 


9 This “spread” of NGSS learning was the ultimate goal of the Early Implementer work and will be the focus of an evaluation 
report to be released in late spring of 2020. 


As they planned Year 5, Early Implementer lead- 


ers in each district began to tailor professional 


learning to the unique needs of their district and 
their teachers. Two primary constraints prompted 


districts to modify their TLCs in Year 5: 


> Availability and cost of substitute teachers. 


Most districts were already facing some degree 
of substitute teacher shortage, and paying for 


substitutes for many teachers is expensive. 
Concern about cost was accentuated by a 


reduction in Initiative funding to districts in 


Year 5 to encourage a transition to district 
self-sufficiency. 


Time out of the classroom. After four years of 
TLCs, district administrators were expressing 
dissatisfaction with the amount of time teach- 
ers had been spending out of their classrooms 


for both Early Implementer as well as other 


district trainings. Project Directors reported 


that some Teacher Leaders themselves were 
reluctant to continue participating in the 


Initiative over concerns about time out of the 


classroom for TLCs.?° 


Only one district chose to not continue with 


any form of lesson study. None of the districts 


continued with the TLCs exactly as they had been 


conducted in Years 1 to 4 (the district variations 


of TLCs in Year 5 are described below in the TLC 
Modifications section). However, three features of 
the original TLC model were preserved in all of the 


districts that chose to continue with TLCs: 


> Common purpose. TLCs fulfilled three 
main goals during the first four years of the 


Vv 


Initiative; those goals were retained to some 
degree in the Year 5 TLCs. The TLCs fostered a 
culture of collaboration where learning anda 


growth mindset were emphasized over perfor- 
mance for both teachers and administrators. 
They reinforced the return of science as a core 
subject. Finally, they provided effective profes- 
sional learning for teachers about the NGSS 
and how to implement them. 


Facilitated group work and discussion. When 
teachers were convened for TLC activities in 
Year 5, trained facilitators led them through 
the protocol at hand. As with TLCs in Years 1 
to 4, teachers were able to ask questions and 
hear tips from individuals with firsthand NGSS 
instruction experience. 


Planning, teaching, and debriefing. All Year 5 
TLC configurations included some form of 
each of the main TLC activities. Importantly, 
review of student work was always part of the 
debriefs. 


TLC Modifications 


Because of the factors described earlier — the 


cost of substitute teachers and the difficulty of 


securing multiple days for teachers to be out of the 


classroom — chances are that most districts will 


not have the resources to carry out TLCs accord- 
ing to the original Early Implementer model. In 
fact, as the districts experienced a reduction in 


grant funding in Year 5, they had to decide what 


modifications to make to their TLCs to meet their 


new realities. 


10 One of these Project Directors, who had redesigned TLCs to accommodate these teacher requests, told evaluators that 
teachers later had a change of heart. This district did not have issues with substitute teachers and could have continued the 


original TLC configuration. 
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The sections below describe the five primary 
modifications that districts made when imple- 
menting their TLCs in Year 5. A vignette in 
Appendix C vividly describes one of the most 
dramatic reconfigurations of TLCs: grade 5 teach- 
ers doing planning, co-teaching, and debriefing of 
a lesson — allina half day! 


Less Time Out of the Classroom 


To reduce the time teachers were spending in 
professional learning versus with their students, 
some districts used existing professional develop- 
ment structures, such as site-based professional 
learning community (PLC) time for planning. 
Protecting teacher time in the classroom meant 
that some TLC activities in most districts took 
place on Saturdays, during professional develop- 
ment days, and after school. One district offered 
TLC professional learning during the summer 
when teachers could co-teach the lessons to 
summer school classes. Other districts held the 
TLC planning during a morning or an afternoon 
within normal schooltime. Indeed, survey data 
indicates that 39 percent of TLC participants in 
Year 5 received some extra pay for their partici- 
pation. Early Implementer funds were used to pay 
teachers for time outside of their regular contract 
hours. In many instances, however, some (but less 
than in Years 1—4) release time was still included 


as part of the TLC implementation." 


Expanding Participation 


A goal of the Initiative from the outset was that 
Early Implementer Teacher Leaders would share 
their growing NGSS expertise with their peers, so 
that all teachers of science in every district would 


eventually benefit from the professional learning 


provided. Districts began to carry out this respon- 
sibility in earnest in Year 5 by inviting “expansion 
teachers" (i.e., teachers who did not participate 

in the Initiative in years 1-4) to TLCs and other 
Initiative professional learning opportunities. 
Most districts offered lesson studies exclusively 
to teachers of science, but some also involved 
resource teachers and special education teachers. 
One district invested extra funds to include every 
K-8 teacher of science as well as English language 
arts (ELA) and special education teachers. Schools 
that previously had been relatively untouched by 
the Initiative because they had had few, if any, 
Teacher Leaders, were assigned Core Teacher 
Leaders to facilitate their TLCs in Year 5. 


Variations in the Planning Component 


More districts retained the original TLC planning 
day model than the teaching day model. Three 
districts developed a conceptual flow during full- 
day planning sessions in Year 5, as all districts 
had done in Years 1 to 4. Two districts retained 
the development of conceptual flows and 5E lesson 
plans but did this in an abbreviated planning day 


of only a morning or afternoon. 


Facilitators in three districts came with a prepared 
phenomenon, learning sequence, or a conceptual 
flow with which the team could develop a brand- 
new lesson, versus having teachers develop every- 
thing from scratch as had been the case in the 
Years 1-4. In three districts, teachers were given 
an existing lesson — often developed by a past TLC 
team — for teachers to learn and adapt. In other 
districts, planning was omitted for some groups 

of teachers. Instead, these teachers observed and 
debriefed demonstration lessons explicitly taught 
to illustrate what an NGSS lesson looks like. 


11 In surveys, only a quarter of TLC participants (26 percent) reported that they had not needed any substitute time to 
participate in any part of their TLC in Year 5. Thirty-eight percent of respondents needed a sub for a full day, 22 percent for 


more than a full day, and 13 percent for less than a full day. 
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In some districts, full, tested lessons or learning 
sequences were provided. In these cases, binders 
of lessons and units were given to all teachers and 
they were provided with professional learning 

to help them implement them. This support was 
usually intended for larger groups of teachers 
from multiple sites. Teachers would convene to 
learn a part of the sequence together, go back to 
their classrooms to teach that lesson or set of 
lessons independently with their own students, 
and then reconvene to learn the next lesson or set 


of lessons in the learning sequence. 


Variations in the Teaching Component 


All of the districts reduced the amount of time 
teachers spent out of their classrooms by dropping 
the formal co-teaching portion of the TLC (with 
the exception of one district's "mini-TLC,” in which 
the planning, teaching, and debrief all occurred in 
half a day — see Appendix C for a vignette of this 
model). Instead of co-teaching, teachers in Year 5 
brought the lesson they had planned or learned 
with other teachers to their classroom and taught 
it to their students. Some individuals managed to 
either observe another teacher teach the lesson 

to their students or informally co-teach with a 
fellow teacher, but this was neither consistent nor 
required.” In districts in which observation of a 
demonstration lesson replaced the experience of 
planning and teaching the lesson, the experience 


was passive rather than active. 


Variations in the Debrief Component 


All modified TLCs in Year 5 included facilitated 
review of student work, which was identified by 


Early Implementer leaders as essential. However, 


some districts scheduled less time for the debrief. 
In three districts, debriefing occurred after 
school, and in two others debriefing occurred on 
Saturdays. One district had debriefing activities 
both after school and on Saturdays. 


Another variation in Year 5 was the nature of the 
work that occurred during the debriefing. When 
there was no lesson revision and reteaching, 

this aspect required less time. All modified TLCs 
in Year 5 still included some teacher review 

of student work with quick scoring or sorting 

of student work into categories such as high, 
medium, and low or “got it” and "didn't get it.” 
Some districts spent debrief time developing a 
rubric as a group. Although the debrief component 
typically included a discussion of what parts of 
the lesson worked and, in some cases, how it could 
be improved, none of the Year 5 district models 
included a formal reteaching of the lesson. 


Other Variations 


Evaluators have anecdotal data that there were 
Teacher Leaders or site leaders who, at the school 
level, tried even more variants of TLC. In one 
school, for example, Teacher Leaders used the TLC 
format to provide site-based NGSS professional 
learning for teachers who had not participated 

in the Initiative and had limited understanding 

of the standards. The facilitators and the partici- 
pants met after school to discuss and plan NGSS 


lessons, as this Core Teacher Leader reported: 


We also had several Teacher Leaders that 
were facilitating their own lesson studies 
at their sites with [expansion] teachers. 


For example, two Teacher Leaders worked 


12 Evaluators heard of a kindergarten Core Teacher Leader in one district who co-planned a lesson with her school 
kindergarten team after school and during their prep period. The kindergarten teacher taught the lesson first and then observed 
and co-taught the lesson in each of the other teachers’ classes. Each teacher had an individual debrief after teaching the lesson, 


and then the team held a debrief together at the end. 
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with the kinder team at [school name]. 
The teachers they were working with had 
very limited NGSS-focused training time 
previously. [Some] of those teachers ... 
stayed after school, working with Teacher 
Leaders on several different days develop- 


ing their lessons. 


Vignette: One District's 
Year 5 Use of TLCs to 
Reach All Teachers 


One Early Implementer district chose TLCs as the 
primary mechanism in its Years 5-6 funding to 
not only implement the NGSS, but also to share 
some of the most effective strategies and tools 
used by the Early Implementers Initiative with 

all teachers, across all subjects. This district had 
enjoyed a high level of district leader and site 
administrator support for science and the NGSS, 
and strong buy-in from the teachers, parents, and 
the community. The district broadened its TLCs 
to include all teachers in grades TK through 5 
(including special education teachers). Further, 
middle school science and ELA teachers engaged 
in TLCs together to develop and teach coordi- 
nated lessons for their grade levels. The following 
vignette elaborates this district's Year 5 TLC 


implementation story. 


Administrative and District-Level 
Support for Science as a Subject and 
for TLCs 


The first key thing that this district did to imple- 
ment its “lesson study for all” model was to 
harness support from its district leaders. Doing 


so allowed them to ensure funding and release 


time for TLCs and created more accountability 
for teachers to participate. Central office support 
also created an environment that encouraged site 
administrators to be more open and supportive to 
the positive impacts that NGSS implementation 


can have on teachers and students. 


The district superintendent had long been 
supportive of the NGSS work that teachers and 
administrators had done and felt strongly about 
building on that capacity: “If we don't capitalize 
on the opportunities the NGSS work provides for 
us, it would be the biggest missed opportunity in 


w 


the history of missed opportunities!" The super- 
intendent’s strong support of science led district 
leadership to send a “clear message to teachers, 
administrators, and community members that 
science is a focus in [the district], one of the three 
[district] strands: science, arts, and languages” 


(Year 4 District Report). 


As a result of strategic efforts by Initiative teach- 
ers and the district Project Director, district 
leaders’ messaging, and firsthand observations 
of NGSS lessons that showed the impacts of the 
NGSS on student engagement, principals in this 
district came to “understand that the NGSS 
provide a space for language and math that can 
assist with the goal of student achievement on 
standardized assessment.... The more adminis- 
trators understand about the NGSS, the more they 
are on board” (Year 4 District Report). 


The combined administrator support at the 
district and school levels resulted in the following: 


> Aunified message that science is important 


> Financial (via both district LCAP and site 
funds) and administrative support for release 
time for all TK through 5 teachers, middle 
school science teachers, and select ELA teach- 
ers to engage in two rounds of TLCs each 
during Year 5 of the Initiative 
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> Designation of two out of the four district- 
controlled PLC professional learning days for 
science TLC debriefs (these days have histor- 
ically been used for initiatives or activities 
other than science) 


> Expectation that all TK through 5 teachers, 
middle school science teachers (and select ELA 
teachers) would attend science TLCs (Year 5 
District Report) 


“Lesson Study for All” Model 


One of the biggest shifts for this Early 
Implementer district in Year 5 centered on 
increasing the capacity and reach of their TLCs. 
They implemented a modified version of the Early 
Implementer TLC, while including as many teach- 
ers in the district as possible. Their model sought 
to accomplish several things: 


> Leverage the leadership capacity and expertise 
of the Early Implementer Teacher Leaders 
and Project Director gained over the first four 
years of the Initiative 


> Reduce the teachers’ time out of the class- 
room, and the need for substitute teachers, by 
convening TLCs during time already desig- 
nated for PLCs 


> Reach all science “expansion teachers” in the 
district (including special education teachers) 
with this targeted and intensive NGSS profes- 
sional learning experience 


> Further collaboration across content areas, 
specifically ELA and science, to build coher- 
ence for students 


> Provide a robust NGSS professional learning 
experience for teachers to “align their efforts 
and share strategies, and to provide a process 
and protocol for analyzing students’ work and 
students’ thinking in a purposeful way” (Year 4 
District Report) 


Figure 2. Graphic overview of an Early 
Implementer district's Year 5 TLC process 
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Source: District TLC Manual (2018-19) 


In Year 5 of the Initiative, this district saw partici- 
pation across all nine K—8 school sites. The district 
also added 115 new “expansion teachers” into their 
NGSS TLCs in Year 5. They had approximately 

4o grade-level TLC teams in the district working 
together to develop NGSS-aligned lessons and 
reflect on their teaching and students’ learning. 


The district's Project Director developed a 

Year 5 TLC manual that broke down each step 
of the process (see Figure 2), provided tools and 
strategies for teachers to use, and ensured that 
teachers were thinking and acting in a way that 
put high-quality teaching of science at the fore- 
front. While their TLCs were still quite rigorous 
and had most of the elements of the TLC used 
in Years 1 to 4 of the Initiative, there were a few 
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modifications made to suit this district's specific 


needs and constraints. 


The primary modification this district needed 
was to reduce the time required for convenings. 
While the district did not want to compromise on 
having a full day for teacher planning, they did 
choose to preserve teacher time in the classroom 
by condensing the teaching and debriefing day. To 
do this, they asked all teachers to teach the TLC 
lesson on their own (though some co-taught with 
colleagues) in their own classrooms before coming 
back together later in the semester for a shared 
debrief session during their PLC time (an early-re- 
lease day). 


Another strategy that this Early Implementer 
district used was inviting all teachers to its 
Summer Institute, which was held immediately 
before the start of the school year. At the Institute, 
grade-specific lessons were developed that teach- 
ers could draw on during TLCs throughout the 
school year. This enabled the district to fully 
leverage the TLC process and ensure practical 
benefits for all teachers (including special educa- 
tion and middle school ELA teachers), while also 
saving some time. During the TLC planning day, 
many teachers would begin by looking at the work 
done in the Summer Institute; further flesh out 
the lesson sequence or unit, or constructing a 
more robust conceptual flow; and then continue to 
collaboratively plan an individual lesson that they 


would all teach in their classrooms. 


This district is making a point of leveraging 
science as a context for engaging students and 
teaching other content, particularly ELA content, 
as recommended by both the NGSS and the 
Common Core State Standards.** The following 
are example topics of integrated science and ELA 


lessons developed by the middle school science 
and ELA teams during their TLCs, drawing on 
the collaborative work done during the Summer 
Institute: 


> Grade 6 
© Science: water distribution and availability 
© ELA: A Long Walk to Water 
> Grade 7 
© Science: ecosystems and engineering 
O ELA: The Martian 
> Grade 8 
© Science: renewable energy 


O ELA: The Boy Who Harnessed the Wind 


These science ideas (taught during science class) 
were reinforced by literature or nonfiction texts 
(taught in ELA class). 


This inclusion of ELA teachers in both 
the Summer Institute work and the TLCs 
has allowed science ideas and content to 
infiltrate other content areas as teachers 
work to plan and revise cross-curricular 
sequences with science as the driver... . 
The coherence, extension, and applica- 
tion of science ideas provided through 
these sequences and those like them in 
middle school ... certainly raise aware- 
ness around the idea of science help- 
ing to do some of the heavy lifting [of 
getting more science to students]. (Year 5 
District Report) 


13. Much has been written about the value of instruction that integrates and coordinates science and ELA. This integrated 
approach is discussed in Early Implementer evaluation report #2, The Synergy of Science and English Language Arts: Means and 
Mutual Benefits of Integration: https://www.wested.org/resources/synergy-of-science-and-english-language-arts/ 
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Overall, the district's modified TLC model worked 
well, and the district cited several successes: 


The “lesson study for all” strategy was 
very successful in moving our expansion 
teachers forward, building on the Summer 
Institute, utilizing the leadership devel- 
oped, beginning the systematization of a 
process for purposeful planning and anal- 
ysis of student work, and helping all to 
get smarter about what NGSS and good 
instruction look like in the classroom. 


(Year 5 District Report) 


Challenges to Lesson Study 
Implementation at the 
Districtwide Scale 


The district's goal was to develop a method for 
TLCs that would be sustainable and effective, even 
beyond the Early Implementers Initiative. As such, 
district staff felt that some further modifications 
would make the process even better during Year 6 
of the Initiative because of a few challenges that 
arose during Year 5. The main challenges the 
district faced within the Year 5 model were the 
following: 


> Calendaring 
> Facilitating 
> Messaging 


Calendaring. By including so many teachers 
across the district (a total of 170) in TLCs, and 
having each teacher participate in two full cycles 
in one year, the calendar was complicated, to 

say the least. While the full-day planning time 
was successfully tackled by timely planning and 
thoughtful arrangements with site administra- 
tors, secretaries, and the personnel department, 


there were some complications with the teaching 
and debrief parts of the lesson study. Because 
teachers were asked to teach the planned lesson 
on their own, there were some instances that 
some teachers did not have time, or just had not 
taught the lesson before the debrief session. This 
meant that these teachers were not able to engage 
in the full lesson study process and reap the bene- 
fits from it. 


In addition, teachers only had two PLC days 

per month during which they could hold their 
debrief sessions, which meant that there were 
many teams debriefing simultaneously across the 
district. This became an issue because the district 
Project Director was often asked to support teams 
during their debrief time (especially in the fall 
during the first round of TLCs), but he physically 
could not be in several places helping multiple 


teams at the same time. 


In an effort to remedy the calendaring challenges 
faced in Year 5, and to ensure the highest quality 
lesson study experience for teachers, in Year 6 
teachers in the district will participate in a single 
TLC modeled after the original version (i.e., a full 
release day for planning and a full release day 

for co-teaching, debriefing, and reteaching). The 
Project Director explained that this will ensure 
that “teachers are able to go through the full TLC 
experience as designed. That way, when we are on 
our own [after the Initiative ends in June 2020], 
we know what it is supposed to look and feel like." 


Facilitating. As mentioned above, TLC teams 

that required additional support from the district 
Project Director or Core Teacher Leaders in the 
district sometimes weren't able to get this support 
in Year 5 because of the constrained calendar. As 

a solution, the district encouraged some teams 

to "self-facilitate” using the lesson study protocol 
and facilitator guide. While in most cases this was 


“effective and resulted in a productive learning 
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experience, in others it was less effective and could 
have contributed to less desirable habits of mind” 
(Year 5 District Report). To remedy this issue, in 
Year 6 the district plans to hold a "facilitator work- 
shop or orientation to clarify the expectations 
around facilitation, resources, etc.” (Year 5 District 


Report). 


Messaging. The last challenge that this district 
faced was related to the messaging that the new 
expansion teachers received regarding the ratio- 
nale and intent of the TLCs. While much of the 
trepidation that first-time lesson study partici- 
pants felt was expected and managed through the 
thoughtful facilitation process in the planning day, 
the debrief days were often weaker than intended: 


While facilitators were ideally very clear 
about the debrief expectations, when left 
to their own devices and teaching the 
lesson in their own classrooms, many 
teachers deviated from or modified the 
lesson, which in some ways invalidated 
the “Study” component of the Lesson 
Study model. The effect could be magni- 
fied when there was no facilitator avail- 
able for the debrief, thus the process lost 
some of its oomph. (Year 5 District Report) 


Despite the above challenges, this Early 
Implementer district deemed its Year 5 TLC effort 
a considerable success and is working to further 
enhance the use of TLCs in Year 6 of the Initiative 
and beyond. "The leadership team decided to 
revert back to the prior, more authentic [TLC] 
model... . This would reduce the number of lesson 
study cycles a team goes through but increase the 
effectiveness by a wide margin” (Year 5 District 


Report). 


Feedback on Year 5 
Modifications 


At the end of Year 5, district Early Implementer 
Project Directors were asked what they felt their 
district had gained and/or lost as a result of 
their Year 5 TLC modifications. Reported gains 
included: 


> Laying the groundwork for sustainability after 
the grant concludes: “Supporting fiscally the 
amount of lesson studies we did made it easier 
to do for Year 6, and hopefully for Year 7 and 
beyond, particularly since we may be adopting 
instructional materials." 


> Providing NGSS professional learning to more 
teachers: “More teachers doing more lesson 
study, getting more practice in implementing 
NGSS, and finding out that it [planning a 
three-dimensional NGSS lesson based ona 
phenomenon] can be done and support of team 
helps.” 


> Training more facilitators: “Leveraging 
Teacher Leadership in the district — Teacher 
Leaders facilitated over 20 teams between 
them." 


> Confirming that teachers with some degree 
of NGSS knowledge can collaboratively plan 
a lesson in a half day: “While it was challeng- 
ing for some of the expansion teachers, the 
vast majority were familiar enough with the 
process and therefore took less time.” 


One Project Director noted that the district had 
gained “a system for looking at student work... 
the kind of approach where we can start to say, 
‘This is how we look at student work in [the 
district],’ and it applies across content areas.” 


All of the Project Directors highlighted the 
co-teaching element of the original TLC as the 
most impactful loss, and they provided a variety of 


reasons: 
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> Loss of quality assurance: “Teachers left to 
their own devices may not have taught the 
lesson as designed, which impacts the work 
they bring to the debrief, and lessens the 
effectiveness of the debrief process.” "Some 
teachers dialed back their expectations.” 


> Loss of collaboration: "Teaching on their own 
in their own classroom robs the teacher of 
the collaborative aspect of the lesson study. 
It can become a little less about the lesson we 
designed and more about individual teachers 
and their quirks/styles/biases.” 


> Loss of evidence for debriefs: "We lost a key 
line of evidence related to the debrief: the 
facilitator transcript of what the teacher said/ 
did and what the kids said/did. That very often 
in the old model was a real eye-opener. It also 
served as corroboration (or not) for what the 
teachers thought had happened.” 


> Loss of depth in discussion: “The debriefs were 
shorter, so less depth was achieved, especially 
if everyone didn't teach the same lesson.” 


> Loss of proximity in time between teaching 
and debriefing: In Years 1 to 4, planning and 
teaching days were scheduled at the start of 
the school year. When teachers taught the 
TLC lessons on their own, “it was tough to 
project out a ‘teach by’ date as well as forecast 
a debriefing day that didn't conflict with other 
Wednesday obligations, staff meetings, ‘teacher 
time,’ other trainings, etc. So, often the teach- 
ing would be far removed from the planning.” 


In surveys, facilitators of TLCs were asked to 
compare the Year 5 lesson study configuration 
to the TLC configuration of Years 1 to 4 of the 
Initiative. About one third (34 percent) rated the 
Year 5 TLC “Worse” or "Much worse,” while only 
22 percent rated it “Better” or "Much better,” as 
shown in Figure 3 on page 27. 


Facilitators were also asked to explain at least one 
specific way the lesson study was better or worse 
than the previous TLC configuration in prior 


years. A total of 129 comments were provided. 
Interestingly, these comments sometimes contra- 
dicted the rating an individual had provided and 
occasionally included ways Year 5 was both better 
and worse. Comments fell into a small number of 


categories, described below. 


Co-teaching was missed in Year 5. Facilitators’ 
most prevalent comment (n=68) mirrored that of 
the Project Directors’: When participants did not 
teach the lesson together, the TLC was less effec- 
tive. Most facilitators only lamented the loss of the 
co-teaching element, but some went on to explain 


the impact on the debrief process. 


[The debrief was worse] without the oppor- 
tunity to teach the lesson from a script 
with other teachers viewing, collecting, 
and evaluating student work, revising the 
lesson, and reteaching. ... I feel like the 
power of the lesson study is lost doing it in 


this revised method. 


When we didn't watch each other teach, 
we ended up having to explain to each 
other how things went and missed the 
nuances. It was even more critical to have 
student evidence, but even then, it was 
hard to tell how instructions may have 


been given slightly differently. 


Not having the opportunity to co-teach 
during a release day teaching session and 
the midday conversations leading toward 
a revised lesson and teaching experience 
was a drawback. Need a release day to 
co-teach and debrief with a facilitator. 
Turning teachers loose after the plan- 
ning is generally less effective, even with 


a solid plan. 
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Figure 3. Facilitator ratings of Year 5 TLCs compared to previous years 


How would you compare this new lesson study configuration to the TLC configuration in prior years 
of the Early Implementers Initiative? 
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Source: Facilitator Post-Lesson Study survey administered by WestEd during the 2018-19 school year. (N=187) 


Those last two parts [making changes to While it is difficult to not watch each 
the lesson and reteaching it] were critical, teacher teach, this format does allow 
and we lost them. It had been very useful individual teachers to adjust parts to the 
to see the strengths and weaknesses in a lesson as they see fit for their students. 


lesson and immediately be able to respond. 

Shorter meeting times meant less depth in 

‘ i ion. M = id th 
Some of the teachers “deviated” some from discussion. Many respondents (n=30) said the new 
configuration felt rushed; 22 of them, concentrated 
in the three districts that held half-day planning 


sessions in Year 5, specifically wanted more time 


the planned lesson so the student work 


was tougher to analyze in a common way. 


for planning. A small number (n=5) mentioned 
Still, a few facilitators (n=4) found an upside to inadequate time for debriefing. 


teachers not teaching collaboratively. 
Having only a half day to plan was VERY 


It allowed more time for teachers to imple- challenging. With new teachers who had 
ment lessons, teachers could make the not been through any training — it was 
lessons their own. very hard to get it done. In fact, we did 


Ca 
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not, and I met with them again later after 


school. 


A full planning day is always better to 
think deeper about the standard and fully 
plan out the lesson sequence instead of 


just parts. 


When debriefing, each teacher came with 
a different way of presenting and gath- 
ering data, where in the past teachers 
would teach together and have the same 


outcome. 


The gain was that each teacher taught the 


lesson in her own classroom, which meant 


We need more time to plan, teach, and we had a greater variety of student work 


reflect together. to review and a wider perspective on how 
the lesson worked with different groups of 


We only had half a day to plan. B 
e only had half a day to plan. Before we srdente 


had a full day. It went well, but teachers 


would like a full day. Some preferred holding TLCs with just teachers 


at their school. Eleven facilitators noted that the 


We were more rushed for time when plan- reason they liked the lesson study configuration 


ning. . .. do fear that new teachers might better in Year 5 was because it was site-based. One 


not have the in-depth understanding of respondent explained that site-level collaboration 
: brought a level of comfort and familiarity that was 
[the lesson planning] process that we had ae 

eneficial: 


gotten in the past though. 
I like that it is at the site level and that 


Larger working groups resulted in richer discus- grade levels get to work together to create 


sions. Two districts brought larger groups of the lessons. We are comfortable with each 


teachers together to review a provided lesson other and more willing to try new things. 


before going back to their classrooms to teach 
We were able to go at our own pace when 

it. Fourteen of the facilitators in this situation : 

k : learning about the standards, CCCs, etc. 

commented that they thought having more partic- 


ipants was a benefit. 


Sending teachers to “NGSS 101" sessions should 
only be a first move of many. The steep learn- 

ing curve to understand and teach the NGSS as 
intended necessitates providing intensive profes- 
sional learning opportunities for teachers, such 
as the TLCs described in this report. Districts 
might send teachers to sessions where they hear 
about the basics and are introduced to the NGSS's 
three dimensions and phenomena-driven instruc- 
tion. While such "NGSS 101" presentations may 
be helpful starters, they are highly unlikely to 
prepare and empower teachers to implement the 
standards in ways that let students experience 
the much stronger learning that the NGSS make 
possible. 


Consider establishing TLCs as professional learn- 
ing that has the power to help teachers success- 
fully transition to NGSS instruction. The changes 
in both content and pedagogy required to teach 
the NGSS are deep. It will therefore take deep 
learning experiences for teachers to understand 
the changes, try them, and gain the confidence 
and experience to succeed. Hopefully this report 
has shown that the benefits of TLCs, even in modi- 
fied versions, are substantial. While TLCs are 
costly, the needs they meet and the benefits they 
deliver are high — that is, they require investment 
that pays dividends. 


Attending to the “collaborative” part of Teaching 
Learning Collaboratives is important. The Early 
Implementers Initiative did not name the lesson 
studies “Teaching Learning Collaboratives” merely 
as a branding exercise. Because the instructional 
changes called for by the NGSS are substantial and 
can be intimidating (particularly for elementary 


teachers), strong attention must be paid to the 


need for a collaborative and nonjudgmental safe 
space for teachers to try new things. Therefore, 
whatever professional learning is attempted, it 
must be well facilitated and establish effective 
norms for and realities of collaboration — includ- 
ing recognizing that collaborative processes 
require time and an environment conducive to 


experimentation and reflection. 


Only using the planning component of TLCs will 
not move the needle very far; the teaching compo- 
nent is essential to professional learning. For cost 
or other reasons, leaders could be tempted to only 
institute the lesson planning component of TLCs. 
Through this component, teachers would get to 
discuss the standards and create new lessons. 
However, the real payoff of bringing teachers 
together for this planning will not happen unless 
participants also come together to teach, then 
debrief, the lessons. The opportunity to share 
perspectives on a lesson’s effectiveness, including 
examining the work done by students during the 
lesson, is powerful. These conversations prompt 
teachers to rethink their initial lesson plans and, 
most importantly, advance their understanding of 
how they and the NGSS can best prompt, facilitate, 
and support student learning. 


Leverage TLCs to facilitate adoption of "NGSS- 
aligned” instructional materials; do not expect 
new curricula to stand in for professional learn- 
ing. With new instructional materials for science 
available for adoption, it may be tempting to 
assume that the need for professional learning 

is reduced. The Initiative has examined available 


curricula and found that none richly encompass 
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three-dimensional, phenomena-driven instruc- 
tion that will meet the needs of all districts. At a 
minimum, the NGSS require place-based learn- 
ing sequences in which materials are suited to 
the local environment. Some Early Implementers 
are centering TLCs on new curriculum, both as 
continued NGSS training and as a way for teach- 
ers to familiarize themselves with the strengths 


and gaps in the materials. Teachers convene to 
discuss a unit, individually try the materials in 
their classrooms, and then regroup to compare 
and analyze how things went. Their conclusions 
about how well the materials fit their district can 
then be applied to the next unit,with new rounds 
of reviewing, teaching, and debriefing.” 


14 Next Gen Toolkit for Instructional Materials Evaluation is a resource developed by BSCS, Achieve, and WestEd’s K-12 
Alliance to support districts in selecting instructional materials that best meet their needs. “Next Gen TIME ... provides 
educators with a transformative professional learning experience highly focused on the NGSS and the implementation of 
high-quality instructional materials.” Trainings are available at county offices of education throughout the state of California. 
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Appendix A. 
Evaluation Methods 


Overview 


This report primarily draws on the following Early Implementer evaluation data sources: 


> Surveys: 


fe) 


Retrospective TLC Survey (2015-16, 2016— 
17) taken by Teacher Leaders at the end of 
the school year 


Facilitator Post-TLC Survey (2016-17, 
2017-18, 2018-19) taken by facilitators 
immediately after a TLC experience 


Teacher Leader Post-TLC Survey (2016-17, 
2017-18, 2018-19) taken by Teacher Leaders 
immediately after a TLC experience 


Survey for K-8 principals, taken at the 
end of the school year by principals having 
at least one participating K-8 teacher of 
science (Teacher Leader and/or expansion 
teacher) at their school (2017-18) 


Spring Survey 2019, taken at the end of the 
year by K-8 teachers of science (including 
both Teacher Leaders and/or expansion 
teachers) 


Spring Survey 2019, taken by principals 


having at least one participating K-8 
teacher of science (including both Teacher 
Leaders and/or expansion teachers) at their 
school (2018-19) 


> Interviews with: 


© Each district Project Director (2015-16, 
2016-17, 2017-18, 2018-19) 


O Twenty-two case study teachers across five 
of the eight participating districts (2016-17, 
2017-18, 2018-19) 


O Two expansion teachers per district 
(2017-18, 2018-19) 


© Two administrators per district (2017-18) 


OQ Two Core Administrators per district 
(2018-19) 


OQ Two School site leaders per district with at 
least one participating teacher (2018-19) 


© Two district administrators per district 
(2018-19) 


> Twenty-seven observations of lesson studies 
over Years 1 through 5 of the Initiative in all 
eight Early Implementer districts 
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Survey Questions Used for 
This Report 


Retrospective TLC Survey (2015-16 & 
2016-17) 


1. How many different schools were represented 
in your TLC group (including yours)? 


2. Who was the primary facilitator of your TLC? 


3. To what extent did the TLC explicitly incor- 
porate the review of student work that was 
produced during the lesson? 


4. To what extent did your TLC experiences 
deepen your understanding of the following 
aspects of NGSS? 


The Science Disciplinary Core Ideas (DCIs) 


How to use Science and Engineering 
Practices (SEPs) to teach science and 
engineering 

c. Howto use Crosscutting Concepts (CCCs) 
to teach science and engineering 


d. How to use a three-dimensional approach 
to help students understand a phenomenon 


e. Howto use the engineering design process 
to develop student understanding of 
science and engineering 

f. How to integrate the science disciplines 
(e.g., physical, earth & space, life) to 
increase student learning 


5. To what extent did your TLC experiences 
empower you to be able to use the following 
project tools and practices on your own (should 
you wish to use them)? 


a. How to use phenomena to drive instruction 
How to look deeply and systemically 
at student work as evidence of student 
understanding 

c. How to use science notebooks to elicit 
student sense-making 

d. How to use Claim, Evidence, and Reasoning 
(CER) to advance student thinking 


6. Please pick the project tool or practice that 


you learned the most about from your TLC 
experiences. 

a. Conceptual Flows 

b. 5E Instructional Model 


Phenomena 


a 9 


Questioning strategies to elicit student 
thinking 

e. Looking at student work with colleagues 
f. Student notebooking 

g. Claim, Evidence, and Reasoning (CER) 


Describe how the TLC experience helped you 
understand this project tool or practice. 


Describe something specific you plan to do 
differently in your classroom next year asa 
result of your TLS experiences. 


Would you recommend the TLC experience to 
other teachers? 


Facilitator Post-TLC Survey (2016-17 
& 2017-18) 


i. 


What about this TLC went particularly well? 
Check all that apply. 


a. Meaningful discussion/aha among 
students during lesson 

b. Meaningful discussion/aha among partici- 
pating teachers during planning or debrief 

c. Teachers deepened their understanding of 
3 dimensional instruction 

d. Teachers practiced effective questioning 
strategies while interacting with students 

e. Effective changes were made to the first 
lesson plan 

f. Student notebooks effectively used for 
sense-making 

g- Student work showed clear evidence of 
learning 

h. Student work showed understanding of 
more than one dimension 
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2. 


3. 


Please explain or elaborate on the above. 


What was challenging about this TLC? Check 
all that apply. 


a. Poor collaboration among participating 
teachers 


b. Students lacked expected prior knowledge 
c. Lesson was not carried out as planned 


d. Lesson did not work as planned (e.g., activ- 
ities took longer than planned, questions 
did not elicit expected student response) 


e. Student work did not show adequate 
evidence of learning 


f. Not enough time to review student work 


Please explain or elaborate on the above. 


Please describe 1 or 2 things you learned from 
this TLC experience. 


Facilitator Post-TLC Survey (2018-19) 


1. 


How many teachers in each grade level partici- 
pated in this lesson study? 


Were all of these teacher participants from the 
same school site? 


Did the lesson study group convene for 
planning (i.e., to discuss how to develop a 
new learning sequence or how to implement/ 
modify an existing learning sequence)? 


For how many hours did the group convene for 
planning? 


For this lesson study, which most closely 
applies to the teaching the group did? 


a. Teachers co-taught the lesson TWO times 
b. Teachers co-taught the lesson ONE time 

c. Teachers taught the lesson on their own 
For how many hours did the group convene for 


teaching? (Please choose “o" if teachers taught 
on their own.) 


Did the group convene to debrief? 


8. 


10. 


For how many hours did the group convene to 
debrief? 


For this lesson study, what did the group do 
during the debrief? (Select all that apply.) 


a. Shared what worked and did not work 
when teaching the lesson 


b. Reviewed or discussed student work 

c. Developed a scoring rubric to review 
student work 

d. Hada facilitated scoring session of student 
work 

e. Discussed how to revise the lesson as a 
group 

f. Revised the lesson as a group 

g. Planned or discussed a new lesson 


h. Other (please specify): 


What about this lesson study went particularly 
well? (Select all that apply.) 


a. Meaningful discussion/learning among 
students during lesson 


b. Meaningful discussion/learning among 
participating teachers during planning or 
debrief 


c. Teachers deepened their understanding of 
3 dimensional instruction 


d. Teachers deepened their understanding 
of instruction that integrates science 
disciplines 

e. Teachers deepened their understanding of 
instruction that integrates engineering 


f. Teachers deepened their understanding of 
instruction that integrates environmental 
education 


g. Teachers practiced effective questioning 
strategies while interacting with students 

h. The group identified potential improve- 
ments to the first lesson plan 

i. Student notebooks were effectively used for 


sense-making 


j. Student work showed clear evidence of 
learning 
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11. 


12. 


13; 


14. 


15. 


16. 


17. 


18. 


k. Student work showed understanding of 
more than one dimension of the NGSS 


Please explain or elaborate on the above. 


i. 


Please describe any noteworthy successes not 
listed above. 


What was challenging about this lesson study? 


a. Nochallenges 


Poor collaboration among participating 
teachers 


1. 


c. Students lacked expected prior knowledge 
d. Lesson was not carried out as planned 


e. Lesson did not work as planned (e.g., activ- 
ities took longer than planned, questions 
did not elicit expected student response) 


f. Student work did not show adequate 
evidence of learning 


g. Not enough time to review student work 


Please explain or elaborate on the above. 


Please describe any noteworthy challenges not 
listed above. 


How would you compare this new lesson study 3. 


configuration to the TLC configuration in prior 
years of the Early Implementers Initiative? 


a. Much better 

b. Better 

c. About the same 
d. Worse 

e. Much worse 

f. Ihave no opinion 


g. ITamnot familiar with the previous TLC 
configuration 


h. N/A (Same configuration as before) 
Please explain at least ONE specific way the 


lesson study was better or worse than the 
previous TLC configuration. 


Please describe 1 or 2 things you learned from 
this lesson study experience. 


Teacher Leader Post-TLC Survey 
(2016-17 & 2017-18) 


Please clearly describe one thing you learned 
from this TLC experience that you plan to take 
back to your classroom. 


Teacher Leader Post-TLC Survey 
(2018-19) 


Did you need a substitute to cover your class 
when you participated in any part of this 
lesson study? (Choose the total amount of time 
a substitute covered your class in order for you 
to participate in all parts of the lesson study.) 


a. Yes, for more than one full day 
b. Yes, for one full day 

c. Yes, for less than a full day 

d. No 


Did you receive any extra pay (e.g., stipend or 
compensation on top of your regular teaching 
salary) for participating in any part of this 
lesson study? 


To what extent did your lesson study experi- 
ence deepen your understanding of the follow- 
ing aspects of the NGSS? 


a. Science Disciplinary Core Ideas (DCIs) 


b. How to use Science and Engineering 
Practices (SEPs) to teach science and 
engineering 

c. Howto use the Crosscutting Concepts 
(CCCs) to teach science and engineering 


d. How to use a three-dimensional approach 
to help students understand a phenomenon 


e. Howto use the engineering design process 
to develop student understanding of 
science and engineering 

f. How to integrate the science disciplines 
(e.g., physical, earth & space, life) to 
increase student learning 


g. Howto address environmental education 
in a science unit (e.g., human impacts) 
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4. To what extent did your lesson study experi- 
ence empower you to be able to use the follow- 
ing project tools and practices on your own 
(should you wish to use them)? 


a. Howto use a conceptual flow of science 
concepts to design and/or teach science 
lessons 


b. How to use the 5E instructional model to 
design and teach lessons 


c. How to use phenomena to drive instruction 


d. How to use questioning strategies 
(e.g., teacher-to-student, student-to- 
student discourse) to develop student 
understanding 


e. How to look deeply and systematically 
at student work as evidence of student 
understanding 


f. How to use science notebooks to elicit 
student sense-making 

g. Howto use Claim, Evidence, and Reasoning 
(CER) to advance student thinking 


h. How to modify a lesson based on student 
work. 


5. Would you recommend the lesson study expe- 
rience to other teachers? 


6. Please choose up to 3 things you learned from 
this lesson study experience that you plan to 
take back to your classroom. 


a. The importance of using a phenomenon for 
student engagement 


b. How to access students’ prior knowledge at 
the beginning of the lesson 

c. Howto create and implement a lesson that 
is student centered 

d. Importance of providing students with 
hands on experiences 

e. Howto use the SEPs or CCCs in my 
teaching 

f. How to promote student-to-student 


discourse and interaction 


g. The importance of building into lessons 
time for student thinking and reflection 


h. The understanding that lessons may not 
work as planned 


i. Other (please specify): 


Survey for K—8 Principals (2017-18) 


1. 


During the 2017-18 school year, how much 
impact did the following have on your under- 
standing of NGSS? 


a. The 2017 Early Implementers Summer 
Institute 


b. TLC Plan Day 
c. TLC Teach Day 


d. Evidence of Learning Protocol (outside of 
TC) 


e. Science Walk-Through 


f. Professional learning (PL) within the 
district 


g. Professional learning (PL) outside of the 
district 


Spring Survey for Science Teachers 
(2018-19) 


1. 


During the 2018-19 school year, how many 
science TLC (lesson study) sessions did you 
attend as a participant, NOT a facilitator? 
Lesson studies are facilitated meetings that 
involve planning, teaching, and/or debriefing 
a specific new or existing lesson with other 
teachers. The debrief sessions often include 
review of student work. 


a. Number of TLC plan sessions you attended 


Number of TLC teach sessions you 
attended 


c. Number of TLC debrief sessions you 
attended 


To what extent did your lesson study experi- 
ence deepen your understanding of the follow- 
ing aspects of the NGSS? 


a. Science Disciplinary Core Ideas (DCIs) 
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How to use Science and Engineering 
Practices (SEPs) to teach science and 
engineering 

How to use the Crosscutting Concepts 
(CCCs) to teach science and engineering 


How to use a three-dimensional approach 
to help students understand a phenomenon 


How to use the engineering design process 
to develop student understanding of 
science and engineering 

How to integrate the science disciplines 
(e.g., physical, earth & space, life) to 
increase student learning 


How to address environmental education 
in a science unit (e.g., human impacts) 


3. To what extent did your lesson study expe- 
rience empower you to be able to use the 
following project tools and practices on your 
own (should you wish to use them)? 


a. 


How to use a conceptual flow of science 
concepts to design and/or teach science 
lessons 


How to use the SE instructional model to 
design and teach lessons 


How to use phenomena to drive instruction 


How to use questioning strategies 
(e.g., teacher-to-student, student-to- 
student discourse) to develop student 
understanding 


How to look deeply and systematically 
at student work as evidence of student 
understanding 


How to use science notebooks to elicit 
student sense-making 

How to use Claim, Evidence, and Reasoning 
(CER) to advance student thinking 


How to modify a lesson based on student 
work. 


4. What leadership role(s) did you have with the 
Early Implementers Initiative this year? (Check 
all that apply.) 


a. 


Site lead at my school 


b. Facilitator of lesson study-related meetings 


c. Facilitator of other professional learning 
for teachers 


d. Core Teacher Leader 
e. Teacher Leader 


Other (please explain) 


Spring Survey for Administrators 
(2018-19) 


1. 


How many science TLC (lesson study) sessions 
did you attend during the 2018-2019 school 
year? 

a. Number of TLC plan sessions you attended 


Number of TLC teach sessions you 
attended 


c. Number of TLC debrief sessions you 
attended 


During the 2018-2019 school year, how much 
impact did the following have on your under- 
standing of NGSS? 


a. The 2018 Early Implementers Summer 
Institute 


b. Lesson Study Plan Session 
c. Lesson Study Teach Session 
Lesson Study Debrief Session 


e. Evidence of Learning Protocol (outside of 
TLC) 


f. Science Walk-Through 

g. Professional learning (PL) within the 
district 

h. Professional learning (PL) outside of the 
district 
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Interview Questions Used for 
This Report 


Project Director Interview #3 (March 

2017) 

i. Another opportunity for leadership growth, 
especially for Core TLs, is the TLC. How do 


you think CTLs are doing in the role of TLC 
facilitator? 


a. How did they prepare to facilitate? 
i. Did each one have similar prep? 
b. Are all CTLs taking on this role? 


i. Ifnot, are some opting to not facilitate 
TLCs or have only a subset of CTLs had 
the opportunity to shadow/facilitate? 


2. Have you been tracking who or how many of 
the K-8 principals have attended/observed 
TLCs? 


a. Optional: Have you gotten feedback from 
them regarding TLCs? 


Project Director Interview #5 (June 
2018) 


1. Is there a story about TLCs in your district? 


a. Will all expansion teachers get same 
exposure/PL? 


b. How is site-based PL dealing with expan- 
sion teachers in grade levels outside the TL 
grade levels? Will they get TLCs? 


Project Director Interview #6 
(October 2018) 


i. Is your district implementing TLCs or lesson 
studies this year? 
a. Ifso, are they site-based or cross-site? 


b. How many have already happened so far 
this year? 


c. Were you in attendance at any of them? 
(Probe: How did the lesson study compare 
to the EI TLCs in years 1-4? What did the 
facilitator do well? What did the facilitator 
struggle with? How would you describe 
the level of participant engagement and 
learning?) 


2. Are lesson studies in your district the same or 
different as the Early Implementer TLCs from 
years 1-4? 


a. If different, what is the primary reason for 
modifying the Early Implementer model? 


3. What are the configurations in your district 
for planning the lesson? (Probe: Where are 
lesson ideas coming from?) 


a. For teaching the lesson? 


b. For debriefing the lesson? (probe re review 
of student work) 


c. Areallsites following the same 
configuration? 


4. Is there any other way that lesson studies in 
your district this year are different from the 
Early Implementer TLCs from years 1-4? 


5. Whois participating in lesson studies? 


a. Are any TLs participating (not as facilita- 
tors) or is it all expansion teachers? 


b. How much variation in levels of partic- 
ipation is there across teachers in the 
district? That is, will some teachers do 
only one lesson study while others will do 
several? 


c. What does the variation across the district 
depend on? 


d. Are all teachers at each site participating 
at the same levels? 


e. Ifnot, what does the variation within a site 
depend on? 


6. Who is facilitating the lesson studies? 


a. Did all of this year’s teacher facilitators 
facilitate a TLC last year? 
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7. 


b. Are any of this year’s facilitators brand new 
to the Early Implementers (i.e., they joined 
the Initiative in spring/summer of 2018)? 


If yes to 5b: What kind of initial training or 
support are inexperienced facilitators getting? 


a. What kind of ongoing support, if any, are 
they getting? 

b. Are they shadowing an experienced facili- 
tator? (Probe: Who are they shadowing: RD, 
PD, CTL, TL? How many times? For one or 
more parts of the lesson study?) 


c. Are experienced facilitators getting any 
ongoing support? 

d. How, if at all, do new and/or experi- 
enced facilitators get feedback on their 
facilitation? 


Is there a push to involve administrators in 
lesson studies this year? 


If administrators are involved, please describe 
how (i.e., what kinds of things are they doing 
during lesson studies, and who is asking them 
to be involved in the lesson studies?) 


Project Director Interview #7 
(January—February 2019) 


1. 


Do you have any updates or corrections to the 
information about lesson studies that was in 
the transcript or the table we sent you? 


Is there any new information from your 
district relating to lesson studies? (For exam- 
ple, are you involving more administrators?) 


(If you want to ask about gaps and clarifica- 
tions, do so now.) I have some clarifying ques- 
tions for you about lesson studies... 


How many lesson studies have you attended 
this year? 


a. What was the best thing you've seen ina 
lesson study? 
b. What have you seen facilitators do well? 


c. What have facilitators struggled with? 


d. How would you describe the level of partic- 
ipant engagement and learning? 


5. Have you hada chance to collect any feedback 


from participants or principals that have 
participated in or observed lesson studies this 
year? 


a. Ifso, can you share it? What was said? 


Some districts are experimenting a bit with 
lesson studies. Are there plans for any new or 
different configuration(s) of lesson study meet- 
ings this year? 


Do you have a plan in mind for lesson studies 
next year? 


Case Study Teacher Interview #1 
(December 2016—January 2017) 


i. 


Have you participated in any additional 
science-related professional development 
events this year or last year that were not part 
of project (e.g., other initiative in the district, 
conference)? 


What are some of the most impactful things 
you have learned from the Early Implementer 
project? How have they affected your 
teaching? 


Case Study Teacher Interview #2 
(June 2017) 


1. 


What EI] events did you participate in since 
our last interview and what are some of the 
most impactful things you learned from them? 
How did they affect your teaching this year? 


Case Study Teacher Interview #3 
(November—December 2017) 


1. 


Have you attended or engaged in any profes- 
sional learning on science since the end of last 
school year? 


a. Ifso, what were they? EII events? 
District/school events? Other events (e.g., 
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conferences)? Other methods of profes- 
sional learning (e.g., online)? 


b. What were the most impactful things you 
learned? 


c. How did or might this PL affect your teach- 
ing this year? 


Case Study Teacher Interview #4 
(May-June 2018) 


1. Were you a teacher participant or a shadower 
or a facilitator? (If facilitator or shadower, how 
many times before had you had that role?) 


a. Ifyes: Tell me about that experience. 


Case Study Teacher Interview #5 
(October-November 2018) 


1. Who will you be mentoring or sharing 
your NGSS expertise with this year, and in 
what context? (Probe whether site-based vs 
district-wide.) 


a. Expansion teachers? Teacher Leaders? 
i. PLCs? 


ii. Lesson studies? 


iii. Other professional learning? 
b. Administrators? 


i. Learning walks? 
ii. Lesson studies? 
iii. Presenting at an admin meeting? 


iv. Other professional learning? 


Case Study Teacher Interview #6 
(April-May 2019) 


i. What training or experience has been the most 
valuable in helping you gain the understanding 
and confidence you need to implement these 
NGSS features? 


Expansion Teacher Interviews #1 and 
#2 (May-June 2017 and May-June 
2018) 


i. 


Have you received formal PD about NGSS from 
your school or district? 


a. Tell me about that (How many times? For 
each: When was it? Who led it? How long 
did it last? How did you find out about it? 
Was it required? Did you get paid to be 
there? Was it grade-specific? Activity or 
presentation?) 

b. What did you think of it? 

c. What did you learn? 


d. Were you empowered or inspired to try 
something new in your classroom as a 
result? If yes, tell me more about that. 


What support or conditions do teachers need 
in order to teach the NGSS? (Note how many 
of these they include: time to plan, time/ 
opportunity to collaborate with other teachers, 
permission to experiment, access to NGSS- 
aligned curriculum/lessons, access to materi- 
als to use in class) 


Has your principal been supportive of you 
learning about or trying out NGSS in your 
classroom? If yes, how so? (Probe examples: 
permission to experiment, provided resources 
or materials, opportunities for training, oppor- 
tunities for collaboration) 


Expansion Teacher Interview #3 
(Spring 2019) 


1. 


What training or experience has been the most 
valuable in helping you gain the understanding 
and confidence you need to implement these 
NGSS features? 


a. What type of training or experience was it? 
(Probe: When was it? Formal or informal? 
Was it required? Did you get paid to be 
there? Was it grade-specific? Activity or 
lecture?) 
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b. Who led it or who shared their expertise 
with you? 

c. Why was it valuable? What did you learn? 
What did you do differently or try in your 
classroom as a result? 


What other training or experience — if any 
— has been valuable in helping you gain the 
understanding and confidence you need to 
implement these NGSS features? 


a. What type of training or experience was it? 
(Probe: When was it? Formal or informal? 
Was it required? Did you get paid to be 
there? Was it grade-specific? Activity or 
lecture?) 


b. Who led it or who shared their expertise 
with you? 


c. Why was it valuable? 


d. What did you learn? What did you do differ- 
ently or try in your classroom as a result? 


What support or conditions do you need in 
order to implement (more) NGSS instruction 

in your classroom? (Note how many of these 
they include: time to plan, time/opportunity to 
collaborate with other teachers, permission to 
experiment, access to NGSS-aligned curricu- 
lum/lessons, access to materials to use in class) 


How has your principal supported your NGSS 
teaching? What has been most valuable? 


Administrator Interview #1 (May—June 
2017) 


1. 


Which of these project events have you attended? 


a. TLCs (Probe: What was your role at the 
event? What do you remember/what did 
you learn from it?) 


Are any of the teachers at your school on the 


CLT? Are there any TLs? If yes: 


a. How many? 


b. What, if any, impact from the Initiative 
have you seen on these individuals? (Probe 


re leadership, instructional practices, 
collaboration.) 


Have you heard other admins talking 
about the impact of NGSS on teachers or 
students? 


Administrator Interview #2 (June—July 


2018) 


1. 


Did you attend any TLCs during the 2017-18 
school year? 


a. 


What was your role at the event? 


i. What specifically did you observe? 

ii. How, if at all, did you participate? 

iii. Did you use an Evidence of Learning 
protocol? (if so, how?) 

What did you learn from this experience? 


How, if at all, did this experience/informa- 
tion received influence your behavior on 
the job? 


How many teachers at your school are partici- 
pating with the Early Implementers grant? 


a. 


What, if any, impact from the Initiative 
have you seen on these individuals this 
year? (Probe re leadership, instructional 
practices, collaboration.) 


Can you give me any specific examples? 


Have you heard other admins talking about 
the impact of NGSS on teachers? 


Core Administrator Interview #1 
(May-July 2019) 


1. 


How many times in all have you observed any 


part of a TLC or lesson study in science? 


a. 


Was it a plan day, a teach day, or just a 
debrief session? 


How many times did you participate in 
some way versus just observe? How did you 
participate (e.g., contributed to the discus- 
sion or debrief)? 
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c. Were any other administrators there and 
did they observe or participate? 


d. Would you say this was effective profes- 
sional learning for you or for the other 
administrators? If no, why not? If yes, what 
did you learn? 


2. What changes have you observed in teachers’ 
ability to teach the NGSS? 


Site Leader Administrator Interview #1 
(May-July 2019) 


1. Have you had any teachers serving as teacher 
leaders in the Early Implementers Initiative at 
your school? 


a. (If yes) About how many Teacher Leaders 
over the course of the Initiative and at 
what grade levels? 


b. (If yes) How, if at all, have they played a 
leadership role at your school? (Probe: 
presented to staff, worked with grade level 
team) 


2. Have you attended any other Early 
Implementer PD specifically for 
administrators? 


a. (If yes) What kind of PD was it? (Probe: 
Admin Academy activity? Presentation at 
an admin meeting? TLC?) 


b. When was it? 


c. Did you feel it was a worthwhile use of 
your time? (If not, what would have made it 
more worthwhile? If so, what stands out in 
your memory about the experience? What 
did you learn?) 

d. Did it impact your behavior at your site? 


(Probe: Did it cause you to do anything new 
to support NGSS implementation?) 


District Administrator Interview #1 
(May-July 2019) 


1. 


Have you attended any NGSS profes- 
sional development in the district for 
administrators? 


a. How many times? (If more than one time) 
Let’s talk about the one that you feel had 
the most impact. 


b. Who led the PD? (Make sure it is connected 
to the Initiative.) 


c. When was it? 


d. What kind of session was it? (How long? 
Districtwide? Site-based? TLC? Summer 
Institute? Walkthrough?) 


e. Who attended? (Which administrators: All? 
Elementary? Middle school?) 


f. What was the purpose? 


g. How did it go? (What worked well? What 
could have been better?) 


h. How, if at all, did it impact administrator 
support for NGSS in the district? 


How, if at all, have Early Implementer partic- 
ipants, such as the teacher leaders or the 
district's Project Director, influenced the 
district's prioritization of science or promotion 
of science as a core subject? 


To what extent would you say the district is 
spreading NGSS implementation to all teach- 
ers (i.e., beyond the Teacher Leaders)? 


a. What strategies have worked best so far to 
advance this effort? 

b. What funding avenues, if any, have been 
explored to support this effort? 

c. What have been the biggest barriers or 
challenges? 

d. What expectations have been communi- 
cated to principals, if any? 

e. What still needs to be done to bring all 
teachers who should be teaching science up 
to speed on the NGSS? Are there plans to 
do this in the near future? 
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Appendix B. Explanation 
of Five Project Tools 


and Practices 


This appendix describes five key tools and prac- 
tices that are central in the professional learning 
provided through the NGSS Early Implementers 
Initiative to teachers and administrators in 

the participating districts. For more detailed 
descriptions of these five tools and practices, 

as well as others, please see the report, Next 
Generation Science Standards in Practice: Tools 
and Processes Used by the California NGSS Early 


Implementers (https://www.wested.org/resources/ 


next-generation-science-standards-in-practice/). 


sense-Making Science 
Notebooks 


Science notebooks are a centerpiece of NGSS 
implementation that Early Implementers have 
been learning about since the first Initiative 
convening. The primary purpose of notebooks is 
for students to record their thoughts, wonderings, 
observations, findings, and evolving understand- 
ing. The secondary purpose is for teachers to see 
evidence of that understanding to help inform 
their planning of classroom activities that elicit 
student thinking and advance student learning. 
Early Implementers learn that the purpose of 
notebooks is not student evaluation. The notebook 


belongs to the student, just as a scientist's note- 


book belongs to the scientist. 


Four “essences” of sense-making notebooks should 


regularly be recorded by the student: 
> Prior knowledge. I think, I predict, I hypothesize 
> Gathering data. I saw, I observed, I measured 


> Making sense of data. Ithink ___ because. . .; 
The data graph shows. . .; Based on the 
evidence... 


> Metacognition. helped my thinking because. . .; 
I wonder. . .; My thinking has changed... 


Questioning Strategies 


Teachers are coached to be very aware of the 

way they facilitate discussion to elicit productive 
student talk. Early Implementers learn question- 
ing strategies to keep instruction inquiry-based, 
so that students are prompted to actively 
construct meaning for themselves. That is, rather 
than provide answers to student questions, teach- 
ers respond with their own thought-provoking 
questions, such as, “What do you think could be 
going on?” “How do you know?” "Do you agree 
with X? Why?” “How could you find out about 
that?” Besides fostering meaningful student 
engagement with the content, this questioning 
strategy encourages increased and improved 


teacher—student and student—student discourse. 
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5E Instructional Model 


Based on the constructivist approach to learning, 
which says that learners build new ideas on top 
of old ideas, the 5E instructional model’ is driven 
by student questioning and discussion. At each 
stage of the lesson — Engage, Explore, Explain, 
Elaborate/Extend, Evaluate — students prac- 
tice and develop literacy skills. They record and 
discuss their prior knowledge of a phenomenon; 
share ideas with peers; conduct investigations; 
read texts, watch video clips, or otherwise take in 
new information; and revise and articulate their 


new thinking. 


Defining Phenomena 


The Early Implementers Initiative defines 
phenomena as “occurrences in the natural and 
human-made world that can be observed and 
cause one to wonder and ask questions.” Focusing 
instruction on phenomena “requires students to 
use the science and engineering practices (SEPs), 
crosscutting concepts (CCCs), and disciplinary 
core ideas (DCIs) in concert to explore, investi- 
gate, and explain how and why phenomena occur” 
(Brown et al., n.d.). The NGSS do not specify which 
phenomena to use in science instruction, “because 
phenomena need to be relevant to the students 
that live in each community and should flow in 


an authentic manner” (California Department of 


Education, 2016). Instead, teachers are encour- 
aged to select phenomena that will engage their 
students, taking into consideration the local 
context as well as student ability levels, interests, 


and previous experiences. 


Looking at Student Work 
in a Teaching Learning 
Collaborative 


In the Teaching Learning Collaborative (TLC) 
process, as teachers design the lesson, they create 
“expected student responses” to teacher prompts. 
These expected student responses are used to 
create the descriptions of high-quality student 
work. After the first time teaching the lesson 
they collaboratively designed, the TLC teams use 
the expected student responses and preliminary 
rubrics that they created based on the expected 
student responses to sort student work into 
performance levels. Groups sort student work by 
high, medium, and low performance. The teams 
then identify the characteristics of the lower-per- 
formance work to see where student learning 
might be improved. For example, if the character- 
istic indicates acommon misunderstanding of a 
science SEP or limited use of a literacy skill, the 
team can discuss and agree upon ways to change 
the instruction to address the misconception. 


15 Originally developed by Biological Sciences Curriculum Study (BSCS; Bybee et al., 2006). 
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Appendix C. Vignette of a 
Year 5 “Mini-TLC’ 


In Year 5 of the Initiative, one of the Early 
Implementer districts developed a “mini-TLC.” 

In this setup, small teams of same-grade teach- 
ers, led by two experienced facilitators, planned, 
co-taught, and debriefed a lesson in a half day. 
The following vignette describes in detail one 
such modified TLC session that was observed by a 
WestEd evaluator during Year 5 of the Initiative. 


Planning 


Three grade 4 teachers met with two Core Teacher 
Leader facilitators early in the morning. The 
session began with an overview of the shifts in 
pedagogy required by the NGSS. For instance, 
facilitators began by reviewing key principles 

in How People Learn (National Research Council, 
2000). The group then examined a 5-E planning 
template, which prompted the group to record the 
phenomenon to be addressed in the lesson and to 
fill out the "What Teacher Does” and "Expected 


Student Response” columns.*® 


In order to expedite the lesson planning process, 
the facilitators had brought in a partially filled out 
template that included a suggested phenomenon: 
earthquakes. They asked the teachers to add to or 
modify the lesson plan. One facilitator noted that 
the lesson plan was, "Purposefully blank — nota 
full-blown SE. Fill in the rest with your expertise.” 
The other facilitator added, “You know your kids 
better than we do, please let us know if there's 
anything you want to change.” 


; cy observations 


hes ared people /kids 


« inside 
+ aly under toy car 
. Firs Shaki 


Top Photo: Materials used during the teaching portion of the 
TLC to simulate an earthquake (e.g., shake table). 


Bottom Photo: Fourth grade students’ observations, 
questions, and connections recorded during the lesson. 


Facilitators and teachers reviewed a series of 
earthquake videos, thinking that students would 
watch one to start the lesson "without sound 

so they focus on what they see.” One facilitator 


16 Fora fuller description of the 5E instructional model and how it is used during TLC planning, see the special Early 
Implementer evaluation report, Next Generation Science Standards in Practice: Tools and Processes Used by the California 


NGSS Early Implementers: https://www.wested.org/resources/next-generation-science-standards-in-practice/ 
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encouraged teachers to think about student 
responses to the video and use that to frame their 
instruction: "If they were writing down their 
observations, what do you think they would say 
they saw? So as we go through this sequence, 
think of how you would word it. How would you 


dissect this lesson as you teach it?” 


Then, they discussed the flow of the lesson: 
Students will watch the earthquake video twice, 
and record their observations, questions, and 
connections. Afterwards, students will engage in 
an “Explore” activity with a shake table that would 
help them simulate an earthquake and observe the 
effects on a building. They will conclude by creat- 
ing a model of what they saw and use it to explain 


properties of waves. 


Throughout the lesson planning portion of the 
TLC, facilitators and teachers engaged in dialogue 
around what students might know (or not know) 
and how that could inform their instruction. 
Referring to students’ background knowledge, a 
teacher expressed some concern that watching a 
video of a Richter scale before this lesson might be 
leading students too much. Another teacher was 
unsure that students would know how to draw a 
wave, so asked, “If kids aren't sure how to make 

a wave, do we ask them to draw a picture of what 
a giant earthquake would look like, andthen a 


smaller earthquake?” 


In addition, facilitators reminded teachers of the 
lesson goals. For example, through the “Engage” 
activity, [Students] become the earthquake. We're 
trying to get kids to understand that there are wave 
properties, amplitude.” When asked by a teacher if 
vocabulary should be reviewed beforehand, a facil- 
itator responded that the activity provides context 
that words can be attached to, and that students 
might understand the “concept but not the word,” 
or vice-versa. The facilitator added, "When they 
get the experience, it gives them the content to 


hopefully make it stick." Then, the "Explain" portion 


of the lesson connects to the NGSS Performance 


Expectation: “The Performance Expectation is 


asking kids to develop a model of waves to see 


patterns of amplitude and wavelength." 


Before heading into the classroom, teachers 


divided up the lesson and discussed who was 


teaching what instructional segments. They 


practiced using the shake table and a facilitator 


encouraged them to manipulate the magnitude of 


the “earthquake” during their instruction: "When 


you model it, show different amounts of energy 


while shaking.” Teachers planned to have students 


work in groups. Students would have different 


roles: shaking the table, holding the paper, grip- 


ping the pencil, and monitoring the time. Students 


would simulate earthquakes first without a build- 


ing and then with a building. 


Teaching 


The teaching portion of the TLC began with 


students watching a video of an earthquake with 


no sound. Then, they watched the video a second 


time and were instructed to write down what they 


saw, specifically their observations, questions, and 


connections. A teacher clarified that the questions 


are their questions, and connections are what 


students thought of as they watched the video. 


Students shared out the following responses anda 


teacher recorded them on chart paper: 


Vv Vv Vv WN VN VY VY 


"I observed they were mostly inside.” 
“Question — what does the earthquake do?” 
“How does the stuff come off? The roof.” 
“Lights. They were like shaking." 

"How do earthquakes form?” 

“How did the power shut down?” 


“Where do earthquakes occur?” 
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> “How long do earthquakes last?" 
> "Why does the earth shake?” 


A teacher categorized students’ responses into 
observations and questions. After many students 
shared out, a teacher asked, "Does anyone have a 
connection?” A student said that earthquakes are 
natural disasters. Another student added that they 
are happening now. A third student stated that 

he did not see someone under the desk but should 
have. After the share-outs, a teacher collected 


student work. 


A teacher modeled how to use the shake table with 
a volunteer and provided instructions for how 
students should record their data. They would 
count the number of spaces every 10 seconds. A 
teacher asked students to “experiment with tiny 
movements and larger movements.” When asked 
what the wave drawing looked like, a student 
responded, “a thing from the hospital.” 


Students in groups of three or four experimented 
with the tables. First, they just used the board and 
then they added blocks to it. On their worksheets, 
students drew the waves with their pencils, as well 
as the arrangement of the blocks on their boards. 


Teachers instructed students to hand in the mate- 
rials and turn over their worksheets. A teacher 
asked students to make a model that showed their 
understanding of what they learned during the 
lesson. Students had the options of recording their 
thinking using words or numbers. Students had 
just a few minutes to draw what they saw and 
what they thought. They were asked to include 


only a few words. 


The Debrief 


One of the facilitators posed the following ques- 
tion at the very beginning of the ten-minute 


debrief: “Based on what you experienced, what 


would you have done differently?” One teacher 
said that at the beginning they should have 
explained a little more about what they would do 
during the lesson. Another teacher thought they 
should have asked students what they know about 
waves. The third teacher responded to this teacher 
by asking, "If they knew we were studying waves, 
would it be a different outcome?” The second 
teacher answered, “But you did say it,” so students 


were aware of the content focus of the lesson. 


The facilitator recommended changing the 
frequency of data recording from 10-second to 
5-second intervals and noted that multiple trials 
with the shake table should be encouraged. 


In terms of what went well, a teacher “was 
impressed [students] worked in groups as well as 
they did,” and another teacher noted good student 
engagement. The first teacher also remarked that 
she appreciated the open-ended nature of the 
lesson because students were free to pursue their 


own wonderings. 


There was a very brief review of student work. One 
of the facilitators asked whether the teachers had 
studied earthquakes already with their students, 
and a teacher responded that they had. 


The teachers said that they would like the facili- 
tators to come back for another TLC. In response, 

a facilitator suggested they talk to their principal, 
as LCAP money had been allocated for science. 
Even though this TLC experience was considerably 
shorter than the TLCs in Years 1—4 of the Initiative, 
it is apparent that it was helpful to teachers in 
planning their NGSS-aligned instruction. 


A possible limitation of the TLC in a nutshell 
experience was that facilitators predetermined 
what standards would be covered and the 
phenomenon that students would experience, 
which prevented TLC participants from develop- 


ing the lesson from scratch. 
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California Science Teachers Association (CSTA) 


— Anonprofit organization dedicated to promot- 
ing high-quality science education and repre- 
senting the interests of science educators at the 
state policy level. Several Early Implementer 
leaders have taken roles in CSTA, and many Early 
Implementer Teacher Leaders have presented 

at the association's annual California Science 


Educator Conference. 


Conceptual Flow — Tool developed by the K-12 
Alliance for mapping the storyline of three-dimen- 
sional NGSS instruction. A conceptual flow can be 
constructed for a six- to eight-week instructional 
unit or a year-long program, depending on the 
complexity of the anchoring phenomenon and how 
many of the grade-level performance expectations 


are incorporated. 


Core Leadership Team (CLT) — Group of 3-5 
administrators and 5-8 teachers established at 
each district at the beginning of the Initiative. The 
CLT meets with their Project Director regularly 
during each school year to plan and lead all Early 
Implementers Initiative activities. They meet 
with their K-12 Alliance Regional Director for six 
Technical Assistance Days each school year. 


Core Teacher Leader (CTL) — Teacher member of 
the Core Leadership Team. Provides professional 
learning to Teacher Leaders, other teachers, and/ 
or administrators in their district or at proj- 


ect-wide events such as the Summer Institute. 


Crosscutting Concepts (CCCs) — One of the three 
NGSS dimensions and a way of linking the differ- 
ent domains of science. CCCs include patterns; 
cause and effect; scale, proportion, and quantity; 


systems and system models; energy and matter; 


structure and function; and stability and change. 


Dimensions of the NGSS — The NGSS includes 
three dimensions: Disciplinary Core Ideas (what 
scientists know), Crosscutting Concepts (how 
scientists make connections among the sciences), 
and Science and Engineering Practices (what 
scientists and engineers do, and how scientific 


knowledge develops). 


Disciplinary Core Ideas (DCIs) — One of the three 
NGSS dimensions. According to National Research 
Council's Framework for K-12 Science Education, 
disciplinary core ideas are the important concepts 
in each of four domains: physical sciences; life 
sciences; Earth and space sciences; and engineer- 


ing, technology, and applications of science. 


Expansion Teacher — Teacher who has not 
directly received significant professional learn- 
ing or support from the Initiative but who is 
benefiting through the shared expertise of those 
who have. In larger districts, expansion teach- 
ers are typically in schools with at least one 
Teacher Leader. 


K-8 NGSS Early Implementers Initiative — 
Six-year Initiative (Summer 2014 to spring 2020) 
supporting implementation of the NGSS by eight 
public school districts and two charter manage- 
ment organizations in California. Developed by 
the K-12 Alliance at WestEd in collaboration with 
the California State Board of Education, California 
Department of Education, and Achieve, the Early 
Implementers Initiative builds capacity of partici- 
pating local education agencies to fully implement 
the NGSS in grades K-8. 
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The K-12 Alliance — A WestEd program of science 
education leaders and professional learning 
providers who plan and deliver all project-wide 


activities for the Early Implementers Initiative. 


Learning Sequence — Three-dimensional (3-D) 
NGSS phenomenon-based instruction lasting 
several lessons. A learning sequence is based on 
an investigative phenomenon and represents part 
of a conceptual flow. Learning sequences can be 


designed using the “SE” instructional model. 


Lesson — Three-dimensional (3-D) NGSS phenom- 
enon-based instruction lasting for a single class 
period, typically 45 to 90 minutes, but potentially 
longer. 


NGSS — A set of K-12 science content standards 
developed by states to improve science educa- 
tion for all students. They are composed of three 
dimensions based on the National Research 


Council's Framework for K-12 Science Education. 


Norms — Agreed-upon productive behaviors 
and mindsets that guide a group when working 


together. 


Phenomena — Natural phenomena are observable 
events that occur in the universe and that we can 
use our science knowledge to explain or predict. 

There are two types of phenomena, anchoring and 


investigative. 


Project Director — District person responsible for 
leading all Early Implementers Initiative activities 
for the district and representing the district at 
monthly Initiative-wide planning meetings with 


Regional Directors. 


Science and Engineering Practices (SEPs) — One 
of the three NGSS dimensions, SEPs are the behav- 
iors that scientists engage in as they investigate 
and build models and theories about the natural 


world and the key set of engineering practices that 


engineers use as they design and build models 
and systems. They include asking questions (for 
science) and defining problems (for engineering); 
developing and using models; planning and carry- 
ing out investigations; analyzing and interpret- 
ing data; using mathematics and computational 
thinking; constructing explanations (for science) 
and designing solutions (for engineering); engag- 
ing in argument from evidence; and obtaining, 


evaluating, and communicating information. 


Summer Institute — Weeklong professional 
learning event held every summer to kick off the 
new Early Implementer school year. In Years 1 
through 4, these were regional, typically one in 
the north and one in the south, and were attended 
by all Initiative participants, some as leaders 
(Regional Directors, Project Directors, Core 
Leadership Team members) and others as learners 
(Teacher Leaders). Beginning in Year 5, a separate 
Summer Institute was held in each district that 
was open to any interested principal or teacher 


of science. 


Teacher Leader (TL) — One of 30-70 teachers in 
each district who joined the Early Implementers 
Initiative in Year 2, one year after the Core Teacher 
Leaders. Teacher Leaders attend annual Summer 
Institutes and participate in two TLCs each school 
year (one in the fall and one in the spring) and 


other district-level professional learning. 


Teaching Learning Collaborative (TLC) — 
Lesson-study activity of Years 1 to 4 of the Early 
Implementers Initiative. Each TLC brings together 
three to four same-grade Early Implementers 
Initiative teachers from different schools within 
the district. Teachers plan and teach a lesson 

to two classrooms of students and debrief after 
each lesson is taught during which they examine 
student work from the lesson. Each Teacher Leader 
participates in two TLCs per year. 
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